topic Re: Nedd regex help to use part of a filepath as source type in Splunk Search

Nedd regex help to use part of a filepath as source type

brent_weaver — Tue, 29 Sep 2020 13:32:42 GMT

I have a source of /var/log/opscode/desired_sourcetype/current. I need to get the part of the filename that is called "desired_courcetype" via regex. I am almost there, the rewriting is working great. Here is my config:

Props:

[chef:server]
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TRANSFORMS-update_metadata = autosource

Transforms:

[autosource]
DEST_KEY = MetaData:Sourcetype
SOURCE_KEY = MetaData:Source
REGEX = \w+
FORMAT = sourcetype::chef:server:$4

Clearly I am not well versed in regex. So woud would the regex be to capture the 3rd element of the filepath delimited by the /.

Any help is MUCH appreciated!

Re: Nedd regex help to use part of a filepath as source type

gcusello — Wed, 05 Apr 2017 13:12:31 GMT

Hi brent_weaver,
do you want to extract this field at search time or at index time?
at search time you could use a regex like this

\/var\/log\/opscode\/(?<desired_courcetype>[^\/]*)

test it at https://regex101.com/r/8YMnMh/1
Bye.
Giuseppe

Re: Nedd regex help to use part of a filepath as source type

brent_weaver — Wed, 05 Apr 2017 13:20:37 GMT

Guiseppe - I would want it at index time. This config is sitting on a heavy weight forwarder. So if I used that config what would my FORMAT field look like in transforms?

Re: Nedd regex help to use part of a filepath as source type

brent_weaver — Wed, 05 Apr 2017 13:41:52 GMT

Also inportant to note that "desired_sourcetype" is variable, that is not a static string.

Re: Nedd regex help to use part of a filepath as source type

somesoni2 — Wed, 05 Apr 2017 13:45:56 GMT

YOu can just use the same REGEX (minus the name capture), i.e. just \/var\/log\/opscode\/([^\/]*) as REGEX in your transforms.conf.

Re: Nedd regex help to use part of a filepath as source type

woodcock — Wed, 05 Apr 2017 13:49:58 GMT

Like this:

[autosource]
SOURCE_KEY = MetaData:Source
REGEX = ^source::(?:/[^/]+){3}/([^/]+)/
FORMAT = sourcetype::chef:server:$1
DEST_KEY = MetaData:Sourcetype

Re: Nedd regex help to use part of a filepath as source type

woodcock — Wed, 05 Apr 2017 13:51:41 GMT

Deploy to Indexers (or HFs) and restart splunk instances and verify on NEW events (old events will stay broken).