props.conf : regex in source stanza

[rashiagrawal]

Hi,

I am trying to filter out events using props.conf and transforms.conf . I have requirement where there are multiple source log files are present and I need to pick few of them , like ( i can't use host or sourcetype as they are been shared with other indexes) -

source1: ABC/DEF/IJK-YTL/master/dev/jobid18/console

source2: ABC/DEF/IJK-YTL/master/dev/jobid19/console

and so on . 

I have tried following regex but they didnt work ( i still see logs been indexed and not dumped) 

props.conf

[source::ABC/DEF/IJK-YTL/master/dev/.*?/console]
TRANSFORMS-set = setnull

OR 

option 2 : [source::\ABC\/DEF\/IJK-YTL\/master\/dev\/.*?\/console]

Option 3 : [source::.../console]
Option 4 : [source::...[/\\]master[/\\]...[/\\]console]

Transforms.conf 

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

Can someone please help - SPlunk is on windows platform

[isoutamo]

Hi

here is told how that should work on props.conf https://docs.splunk.com/Documentation/Splunk/latest/Admin/Propsconf

Have you already try this

[source::...ABC/DEF/IJK-YTL/master/dev/jobid*/console]

Basically this is same than your Option 3. 

Remember to restart instance after change props.conf!

r. Ismo 

[rashiagrawal]

Yes, I am restarting Splunk service after making the change in config files. 

Still no luck !

[isoutamo]

Your configurations are in indexer or if there is heavy forwarder between source system and indexer then on that HF?

Are both Indexer and UF windows nodes? If so then I think that your file url should use windows notations for path \ not /.

r. Ismo

[rashiagrawal]

I found issue with the file name - it was "transform" instead of "transforms".

This option worked like a charm : 

Option 3 : [source::...console]

[isoutamo]

Good to hear that this is solved. Be aware that using just …/console can match many different files later on and then it could be anew challenge to find the reason….