topic Dynamic sourcetype based on source not working in Getting Data In

Dynamic sourcetype based on source not working

davebo1896 — Fri, 03 Jun 2016 18:43:35 GMT

I'm trying to set sourcetype based on a regex from the source path during indexing, and it is not working.
What am I doing wrong?

props.conf

[source::/var/log/docker/...]
TRANSFORMS-setsourcetype = setsourcetype

transforms.conf

[setsourcetype]
SOURCE_KEY = source
REGEX = ^\/var\/log\/docker\/[^\/]*\/([^\/]*)
FORMAT = sourcetype::$1
DEST_KEY = MetaData:Sourcetype

Re: Dynamic sourcetype based on source not working

somesoni2 — Fri, 03 Jun 2016 20:09:43 GMT

The SOURCE_KEY should be MetaData:Source.

Ensure, the configurations are deployed in first full SPlunk instance (heavy forwarder OR indexer)

If it still doesn't work, try [source::/var/log/docker/*] instead of [source::/var/log/docker/...]

Re: Dynamic sourcetype based on source not working

davebo1896 — Fri, 03 Jun 2016 21:01:45 GMT

I changed props to [source::/var/log/docker//] and verified it is working by adding a SEDCMD.

SOURCE_KEY = MetaData:Source is not working in transforms.conf (I think that should technically be the correct solution, though)

I'll put in a ticket with Splunk support and report back here what we find out.

Re: Dynamic sourcetype based on source not working

davebo1896 — Tue, 29 Sep 2020 09:51:44 GMT

Looks like there may be stray characters on the value of source. I added global matches on either end of the source value and now it is parsing fine.

props.conf
[source::/var/log/docker//]
TRANSFORMS-setsourcetype_from_source = setsourcetype_from_source

transforms.conf:
[setsourcetype_from_source]
DEST_KEY = MetaData:Sourcetype
SOURCE_KEY = MetaData:Source
REGEX = .*\/var\/log\/docker\/[^\/]+\/([^\/]+).*
FORMAT = sourcetype::$1