Archive
Highlighted

create multiple sourcetypes from single syslog source

Builder

Hi I'm looking to create events for syslog data from a wireless controller - and the syslog data also contains data from the AP's which is what i'm more interested in.

If i use the generic syslog sourcetype and fix the timestamp then i get all the required fields broken out for the controllers but not all the required ones for the AP's.

how best to create multiple sourcetypes in this instance ? or is there another way ?

thanks.

Blockquote

3/31/17
9:39:06.000 AM

2017-03-31 09:39:06 Local5.Notice X.X.X.X X.X.X.X stm[1234]: <1234> |APName@X.X.X.X stm| Deauth from sta: MACAddy: AP X.X.X.X-MACAddy-APName Reason STA has roamed to another AP

3/31/17
9:39:06.000 AM

2017-03-31 09:39:06 Local5.Notice X.X.X.X MC-Name stm[1234]: <1234> <1234> Assoc success @ 09:39:06.770727: MACAddy: AP X.X.X.X-MACAddy-APName

Blockquote

Tags (1)
0 Karma
Highlighted

Re: create multiple sourcetypes from single syslog source

SplunkTrust
SplunkTrust

Esky73,
You can accomplish this through the use of props / transforms. The general idea of what needs to be done is:
props.conf:
Create a TRANSFORMS- entry under the stanza "[syslog]" that calls a transforms.conf entry that contains a regex to re-sourcetype your events.
transforms.conf:
Create a stanza that's refrenced in props.conf containing a REGEX to match your events, DEST_KEY to set the destination of "FORMAT" to MetaData:Sourcetype and a FORMAT that sets the sourcetype to the desired new sourcetype. See example below.

For this data, my best guess would be something like this:
props.conf:

[syslog]
TRANSFORMS-rewritewirelesscontrollerst = rewritewirelesscontrollerst

transforms.conf:

[rewritewirelesscontrollerst]
REGEX = \S+\s+stm[\d+]:\s+
DEST
KEY = MetaData:Sourcetype
FORMAT = sourcetype::mynewst

This should re-sourcetype your logs to "mynewst". Obliviously change that to whatever makes sense for your environment. The last thing to consider is that these configurations need to exist on the first full Splunk instance that sees this data. In a lot of environments this is the indexer but if you have Heavy Forwarders, then this configuration needs to be there.

Lastly, if you need to further split this, you have the technique, just use additional transforms entries with regular expressions that fit the specific subset of data.

Hope this helps!

View solution in original post

Highlighted

Re: create multiple sourcetypes from single syslog source

Builder

Thanks for your help

** EDIT **

After testing this on a file snippet i took from a live syslog system in which it worked - i then applied the same config to my live environment only for it not to work and not understanding why.

Then i saw the following in the splunk docs :

http://docs.splunk.com/Documentation/Splunk/latest/Data/Bypassautomaticsourcetypeassignment

"Overrides only work on file and directory monitoring inputs or files you have uploaded. You cannot override the source type on network inputs. Additionally, overrides only affect new data that arrives after you set up the override. To correct the source types of events that have already been indexed, create a tag for the source type instead."

which seems to suggest this will not work in my instance as we are using a network input for syslog ?

0 Karma
Highlighted

Re: create multiple sourcetypes from single syslog source

Builder

OK So it does work 🙂

My regex needed to be slightly different in my live env and it seems if one regex is incorrect in the transforms.conf it also stops the other sourcetype overrides from working.

hope this helps someone

thanks again.

0 Karma