How to configure Splunk to distinguish different types of logs from syslog and save them in different indexes on a Splunk Indexer?

[charlescywong]

I have just installed a Splunk App where the logs are from some appliances, so obviously they are sending syslog to our Splunk Indexer. However, this Splunk App is only suitable for analyzing specific logs of one of the vendor appliances (vendor specific Splunk App). So, the problem is that I configured all my network devices (i.e. switch, router, firewall, etc) to send syslog to our Splunk Indexer where the Splunk Indexer stores these logs in the "main" Index.

So, how do I tell the newly installed Splunk App to only capture the "needed" logs to perform analysis from the pool of logs in the "Main" index? (The "need to analyze" logs are in the pool of logs stored in "Main" Index)

[MuS]

Hi charlescywong,

another approach is to re-write the sourcetype of your syslog inputs based on the hostname in the events. This will allow you to send everything over syslog and still have those vendor specific syslog source types in the events.

You can achieve this by using props.conf and transforms.conf on your Splunk instance which does the parsing (Either an Indexer or a heavy weight forwarder).

This example will re-write the sourcetype for events which were send from the host CiscoNetWorkDevice to be cisco_syslog:

props.conf:

[syslog]
TRANSFORMS-001_syslog_set_sourcetype = 001_syslog_set_sourcetype

transforms.conf:

[001_syslog_set_sourcetype]
SOURCE_KEY = MetaData:Host
REGEX = CiscoNetWorkDevice
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::cisco_syslog

Hope this helps and you can get this working ...

cheers, MuS

[charlescywong]

Does this mean re-direct the syslog from CiscoNetworkDevice to cisco_syslog while data is getting into the Splunk Indexer?

[MuS]

It's not really re-direct, the sourcetype will be changed from syslog to cisco_syslog but only for events from host CiscoNetWorkDevice everything else stays sourcetype syslog. You can setup multiple transforms to re-wirte the source types for other devices as well.
Done that multiple times and works perfect.

[ddrillic]

The categorization of data is done via the index and the sourcetype. If you deal with syslogs, you better check whether your logs conform to the pretrained syslogs described at List of pretrained source types