Splunk Search

Using source:: for field extraction in props.conf

Builder

Hi,
I'm trying to extract the name of the tomcat instance based on the path of the source. I've been successful by specifying the sourcetype in props.conf:

[app_foo.log]
EXTRACT-tomcat_instance = /opt/tomcat/(?<tomcat_instance>[^/]+)/logs/.* in source

The above works, but I would like to match all the logs in tomcat directories, since there are several sourcetypes and I'd rather not repeat the same regex several times. So I tried the following:

[source::/opt/tomcat/[^/]+/logs/.*]
EXTRACT-tomcat_test = /opt/tomcat/(?<tomcat_instance>[^/]+)/logs/.* in source

However, this does not seem to work. I've tested the regex with the commands rex and regex and it works there. Any pointers would be appreciated.

0 Karma
1 Solution

Legend

The issue is probably not with the extraction itself, but the stanza you're using - it will not be matched.

The regular expression you can use in the source:: stanza is not the same as the one used by for instance rex - rather it is just a small subset and is even a bit different to "normal" regular expressions. From props.conf.spec:

When setting a [<spec>] stanza, you can use the following regex-type syntax:
... recurses through directories until the match is met.
*   matches anything but / 0 or more times.
|   is equivalent to 'or'
( ) are used to limit scope of |.

So what you want is to use "*":

[source::/opt/tomcat/*/logs]
EXTRACT-tomcat_test = /opt/tomcat/(?<tomcat_instance>[^/]+)/logs/.* in source

View solution in original post

Legend

The issue is probably not with the extraction itself, but the stanza you're using - it will not be matched.

The regular expression you can use in the source:: stanza is not the same as the one used by for instance rex - rather it is just a small subset and is even a bit different to "normal" regular expressions. From props.conf.spec:

When setting a [<spec>] stanza, you can use the following regex-type syntax:
... recurses through directories until the match is met.
*   matches anything but / 0 or more times.
|   is equivalent to 'or'
( ) are used to limit scope of |.

So what you want is to use "*":

[source::/opt/tomcat/*/logs]
EXTRACT-tomcat_test = /opt/tomcat/(?<tomcat_instance>[^/]+)/logs/.* in source

View solution in original post

Engager

Thanks a lots, you saved my lots of time.

0 Karma

Legend

No, I actually removed it explicitly! Splunk will grab everything in the directory that is specified automatically anyway.

Builder

Thanks! That solved it. Btw, don't you mean "[source::/opt/tomcat//logs/]" with a star at the end?

0 Karma
Don’t Miss Global Splunk
User Groups Week!

Free LIVE events worldwide 2/8-2/12
Connect, learn, and collect rad prizes
and swag!