I have multiple whitelists in the same directory. How to edit inputs.conf to make sure it captures all the logs?

[ejwade]

I have all my switch and linux syslogs stored in a single directory - let's call it /var/log/syslog. I'm trying to assign specific indexes and sourcetypes based on a regex statement of the filename. Here's what I have:

[monitor:///var/log/syslog]
whitelist = \/var\/log\/syslog\/^sw.*
disabled = false
index = switches
host_segment = 4
sourcetype = switch_syslog

[monitor:///var/log/syslog]
whitelist = \/var\/log\/syslog\/^[^s][^w].*
disabled = false
index = linux
host_segment = 4
sourcetype = linux_syslog

These are two monitoring stanzas of the same directory. One is pulling out filenames that begin with "sw", and the other is pulling out filenames beginning with anything but "sw". I'm able to capture some of the logs, but not all. Can you see a better way to achieve the same thing?

Thank you.
Ed

[alemarzu]

Hi there, I haven't tested it but this migth guide you.

Create a single input between those two.
inputs.conf

[monitor:///var/log/syslog]
whitelist = (\/var\/log\/syslog\/^sw.*|\/var\/log\/syslog\/^[^s][^w].*)
disabled = false
host_segment = 4
sourcetype = syslog_routing

props.conf

[syslog_routing]
TRANSFORMS-convert_to_sourcetypes = convert_to_switch_syslog, convert_to_linux_syslog
TRANSFORMS-route_by_index = move_to_index_switch, move_to_index_linux

transforms.conf

 [convert_to_switch_syslog]
 SOURCE_KEY = MetaData:Source
 REGEX = ^source::\/var\/log\/syslog\/^sw.*
 DEST_KEY = MetaData:Sourcetype
 FORMAT = sourcetype::switch_syslog

 [convert_to_linux_syslog]
 SOURCE_KEY = MetaData:Source
 REGEX = ^source::\/var\/log\/syslog\/^[^s][^w].*
 DEST_KEY = MetaData:Sourcetype
 FORMAT = sourcetype::linux_syslog

 [move_to_index_switch]
 SOURCE_KEY = MetaData:Sourcetype
 DEST_KEY = _MetaData:Index
 REGEX = switch_syslog
 FORMAT = switches

 [move_to_index_linux]
 SOURCE_KEY = MetaData:Sourcetype
 DEST_KEY = _MetaData:Index
 REGEX = linux_syslog
 FORMAT = linux

[somesoni2]

Splunk will be applying monitoring based on only one monitoring stanza as it merges all configurations available on the forwarder (all splunk instances, just using forwarder as an example), probably the last one. You can just have one monitoring stanza with combined whitelist (merge regex) and then use props-transforms on the indexer/heavy forwarder to override the metadata fields. See this for later part.

https://answers.splunk.com/answers/301504/how-to-override-sourcetype-and-index-assignment.html