topic Re: props.conf : regex in source stanza in Splunk Search

props.conf : regex in source stanza

rashiagrawal — Wed, 12 Jan 2022 05:16:10 GMT

Hi,

I am trying to filter out events using props.conf and transforms.conf . I have requirement where there are multiple source log files are present and I need to pick few of them , like ( i can't use host or sourcetype as they are been shared with other indexes) -

source1: ABC/DEF/IJK-YTL/master/dev/jobid18/console

source2: ABC/DEF/IJK-YTL/master/dev/jobid19/console

and so on .

I have tried following regex but they didnt work ( i still see logs been indexed and not dumped)

props.conf

[source::ABC/DEF/IJK-YTL/master/dev/.*?/console]
TRANSFORMS-set = setnull

option 2 : [source::\ABC\/DEF\/IJK-YTL\/master\/dev\/.*?\/console]

Option 3 : [source::.../console]
Option 4 : [source::...[/\\]master[/\\]...[/\\]console]

Transforms.conf

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

Can someone please help - SPlunk is on windows platform

Re: props.conf : regex in source stanza

isoutamo — Wed, 12 Jan 2022 10:25:18 GMT

here is told how that should work on props.conf https://docs.splunk.com/Documentation/Splunk/latest/Admin/Propsconf

Have you already try this

[source::...ABC/DEF/IJK-YTL/master/dev/jobid*/console]

Basically this is same than your Option 3.

Remember to restart instance after change props.conf!

r. Ismo

Re: props.conf : regex in source stanza

rashiagrawal — Wed, 12 Jan 2022 15:31:19 GMT

Yes, I am restarting Splunk service after making the change in config files.

Still no luck !

Re: props.conf : regex in source stanza

isoutamo — Thu, 13 Jan 2022 06:41:47 GMT

Your configurations are in indexer or if there is heavy forwarder between source system and indexer then on that HF?

Are both Indexer and UF windows nodes? If so then I think that your file url should use windows notations for path \ not /.

r. Ismo

Re: props.conf : regex in source stanza

rashiagrawal — Thu, 13 Jan 2022 15:59:56 GMT

I found issue with the file name - it was "transform" instead of "transforms".

This option worked like a charm :

Option 3 : [source::...console]

Re: props.conf : regex in source stanza

isoutamo — Thu, 13 Jan 2022 21:06:46 GMT

Good to hear that this is solved. Be aware that using just …/console can match many different files later on and then it could be anew challenge to find the reason….