Splunk Search

Using source:: for field extraction in props.conf

echalex
Builder

Hi,
I'm trying to extract the name of the tomcat instance based on the path of the source. I've been successful by specifying the sourcetype in props.conf:

[app_foo.log]
EXTRACT-tomcat_instance = /opt/tomcat/(?<tomcat_instance>[^/]+)/logs/.* in source

The above works, but I would like to match all the logs in tomcat directories, since there are several sourcetypes and I'd rather not repeat the same regex several times. So I tried the following:

[source::/opt/tomcat/[^/]+/logs/.*]
EXTRACT-tomcat_test = /opt/tomcat/(?<tomcat_instance>[^/]+)/logs/.* in source

However, this does not seem to work. I've tested the regex with the commands rex and regex and it works there. Any pointers would be appreciated.

0 Karma
1 Solution

Ayn
Legend

The issue is probably not with the extraction itself, but the stanza you're using - it will not be matched.

The regular expression you can use in the source:: stanza is not the same as the one used by for instance rex - rather it is just a small subset and is even a bit different to "normal" regular expressions. From props.conf.spec:

When setting a [<spec>] stanza, you can use the following regex-type syntax:
... recurses through directories until the match is met.
*   matches anything but / 0 or more times.
|   is equivalent to 'or'
( ) are used to limit scope of |.

So what you want is to use "*":

[source::/opt/tomcat/*/logs]
EXTRACT-tomcat_test = /opt/tomcat/(?<tomcat_instance>[^/]+)/logs/.* in source

View solution in original post

Ayn
Legend

The issue is probably not with the extraction itself, but the stanza you're using - it will not be matched.

The regular expression you can use in the source:: stanza is not the same as the one used by for instance rex - rather it is just a small subset and is even a bit different to "normal" regular expressions. From props.conf.spec:

When setting a [<spec>] stanza, you can use the following regex-type syntax:
... recurses through directories until the match is met.
*   matches anything but / 0 or more times.
|   is equivalent to 'or'
( ) are used to limit scope of |.

So what you want is to use "*":

[source::/opt/tomcat/*/logs]
EXTRACT-tomcat_test = /opt/tomcat/(?<tomcat_instance>[^/]+)/logs/.* in source

rikin_patel
Engager

Thanks a lots, you saved my lots of time.

0 Karma

Ayn
Legend

No, I actually removed it explicitly! Splunk will grab everything in the directory that is specified automatically anyway.

echalex
Builder

Thanks! That solved it. Btw, don't you mean "[source::/opt/tomcat//logs/]" with a star at the end?

0 Karma
Get Updates on the Splunk Community!

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...

Splunkbase | Splunk Dashboard Examples App for SimpleXML End of Life

The Splunk Dashboard Examples App for SimpleXML will reach end of support on Dec 19, 2024, after which no new ...

Understanding Generative AI Techniques and Their Application in Cybersecurity

Watch On-Demand Artificial intelligence is the talk of the town nowadays, with industries of all kinds ...