Hi I'm looking to create events for syslog data from a wireless controller - and the syslog data also contains data from the AP's which is what i'm more interested in.
If i use the generic syslog sourcetype and fix the timestamp then i get all the required fields broken out for the controllers but not all the required ones for the AP's.
how best to create multiple sourcetypes in this instance ? or is there another way ?
2017-03-31 09:39:06 Local5.Notice X.X.X.X X.X.X.X stm: <1234>
2017-03-31 09:39:06 Local5.Notice X.X.X.X MC-Name stm: <1234> <1234>
You can accomplish this through the use of props / transforms. The general idea of what needs to be done is:
Create a TRANSFORMS- entry under the stanza "[syslog]" that calls a transforms.conf entry that contains a regex to re-sourcetype your events.
Create a stanza that's refrenced in props.conf containing a REGEX to match your events, DEST_KEY to set the destination of "FORMAT" to MetaData:Sourcetype and a FORMAT that sets the sourcetype to the desired new sourcetype. See example below.
For this data, my best guess would be something like this:
TRANSFORMS-rewritewirelesscontrollerst = rewritewirelesscontrollerst
REGEX = \S+\s+stm[\d+]:\s+
DESTKEY = MetaData:Sourcetype
FORMAT = sourcetype::mynewst
This should re-sourcetype your logs to "mynewst". Obliviously change that to whatever makes sense for your environment. The last thing to consider is that these configurations need to exist on the first full Splunk instance that sees this data. In a lot of environments this is the indexer but if you have Heavy Forwarders, then this configuration needs to be there.
Lastly, if you need to further split this, you have the technique, just use additional transforms entries with regular expressions that fit the specific subset of data.
Hope this helps!
Thanks for your help
** EDIT **
After testing this on a file snippet i took from a live syslog system in which it worked - i then applied the same config to my live environment only for it not to work and not understanding why.
Then i saw the following in the splunk docs :
"Overrides only work on file and directory monitoring inputs or files you have uploaded. You cannot override the source type on network inputs. Additionally, overrides only affect new data that arrives after you set up the override. To correct the source types of events that have already been indexed, create a tag for the source type instead."
which seems to suggest this will not work in my instance as we are using a network input for syslog ?
OK So it does work 🙂
My regex needed to be slightly different in my live env and it seems if one regex is incorrect in the transforms.conf it also stops the other sourcetype overrides from working.
hope this helps someone