Splunk Enterprise

How to use Regular Expression in props.conf for source matching?

cameronjust
Path Finder

Hi All,

I've got a generic syslog app which pulls in EVERYTHING in the syslog directory with the sourcetype=syslog-unconfigured

inputs.conf

 

 

[monitor:///var/log/syslog-ng/*/messages]
index = syslog
sourcetype = syslog:unconfigured
host_segment = 4

 

 

 

This is done so we can catch any new syslog devices that were not configured to go to the correct sourcetype. We have a props.conf that routes data to the right index/sourcetype depending on the hostname.

props.conf

 

 

# InfoBlox
[source::/var/log/syslog-ng/(10.164.55.55|10.9.55.56|prodinfoblox1|prodinfoblox2)/messages]
TRANSFORMS-reroute_index = route_to_index_infoblox
TRANSFORMS-reroute_sourcetype = route_to_sourcetype_infoblox:file
TZ=UTC

 

 

transforms.conf

 

 

[route_to_index_infoblox]
REGEX = .
DEST_KEY = _MetaData:Index
FORMAT = infoblox

[route_to_sourcetype_infoblox:file]
REGEX = .
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::infoblox:file

 

 

Now the above props.conf with a regex for matching on the host in the source doesn't work. However naming each individually does or with a basic wildcard

 

 

# InfoBlox
[source::/var/log/syslog-ng/10.164.55.55/messages]
TRANSFORMS-reroute_index = route_to_index_infoblox
TRANSFORMS-reroute_sourcetype = route_to_sourcetype_infoblox:file
TZ=UTC

[source::/var/log/syslog-ng/10.9.55.56/messages]
TRANSFORMS-reroute_index = route_to_index_infoblox
TRANSFORMS-reroute_sourcetype = route_to_sourcetype_infoblox:file
TZ=UTC

[source::/var/log/syslog-ng/prodinfoblox*/messages]
TRANSFORMS-reroute_index = route_to_index_infoblox
TRANSFORMS-reroute_sourcetype = route_to_sourcetype_infoblox:file
TZ=UTC

 

 

I've tried escaping the slashes but that doesn't work either.

 

 

# This also doesn't work 
[source::\/var\/log\/syslog-ng\/(10.164.55.55|10.9.55.56|prodinfoblox1|prodinfoblox2)\/messages]

 

 

 

Anyone have any ideas how to get the regex to work in the source:: stanza?

Some of these devices have up to 30 hosts and having it all as a one liner would make things much cleaner.

Also I'm aware I can do this in transforms.conf with something like this but then I'd need the source match in two spots which is prone to user error.

 

 

 

 

[route_to_index_infoblox]
SOURCE_KEY = Metadata:Source
REGEX = \/var\log\syslog\/(192.168.1.1|192.168.1.2|etc.)
DEST_KEY = _MetaData:Index
FORMAT = infoblox

[route_to_sourcetype_infoblox:file]
SOURCE_KEY = Metadata:Source
REGEX = \/var\log\syslog\/(192.168.1.1|192.168.1.2|etc.)
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::infoblox:file

 

 

 

 

 

There has to be something just slightly off with my regex.

Labels (1)
Tags (2)
0 Karma
1 Solution

isoutamo
SplunkTrust
SplunkTrust

Hi

if I understand this https://docs.splunk.com/Documentation/Splunk/latest/Data/Specifyinputpathswithwildcards#Wildcard_ove... right, you must use also wildcard "*" if you want to use regular expression style matching. Without it it don't recognise this stanza as a regex.

So can you try this

[source::/var/log/syslog-ng/(10.164.55.55|10.9.55.56|prodinfoblox1|prodinfoblox2)*/messages]

r. Ismo 

View solution in original post

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

if I understand this https://docs.splunk.com/Documentation/Splunk/latest/Data/Specifyinputpathswithwildcards#Wildcard_ove... right, you must use also wildcard "*" if you want to use regular expression style matching. Without it it don't recognise this stanza as a regex.

So can you try this

[source::/var/log/syslog-ng/(10.164.55.55|10.9.55.56|prodinfoblox1|prodinfoblox2)*/messages]

r. Ismo 

0 Karma

cameronjust
Path Finder

IT WORKS!!!

Thanks for the clarification. I do recall I saw this a few years ago but it obviously never stuck in my mind. I certainly won't forget it now.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...