How to ceate a regex blacklist in inputs.conf with a common string value in log file?

[sreesuresh545]

Hi All,

Having issue in identifying the correct blacklist regex expression to skip the few logs which are loading to Splunk.

Below is my monitoring path which is updated in the inputs.conf file:

[monitor:///project-abc/src/logs/*.log]

I will have the log files will be created daily as below:

/project-abc/src/logs/interestrate_runner_2021-04-23_15:06:05.123456_3456789012345_7890.log /project-abc/src/logs/contractrate_runner_2021-05-21_16:06:05.654321_2345345678901_7891.log /project-abc/src/logs/savingscost_runner_2021-05-21_17:08:05.214356_2345345678901_7892.log /project-abc/src/logs/interestrate_2021-04-23_15:06:05.123456_3456789012345_7890.log
/project-abc/src/logs/contractrate_2021-05-21_16:06:05.654321_2345345678901_7891.log
/project-abc/src/logs/savingscost_2021-05-21_17:08:05.214356_2345345678901_7892.log

I want to blacklist the below files:

/project-abc/src/logs/interestrate_runner_2021-04-23_15:06:05.123456_3456789012345_7890.log /project-abc/src/logs/contractrate_runner_2021-05-21_16:06:05.654321_2345345678901_7891.log /project-abc/src/logs/savingscost_runner_2021-05-21_17:08:05.214356_2345345678901_7892.log

I have tried the below regex, but none of them worked.

[monitor:///project-abc/src/logs/*.log]
blacklist = .*runner.*\.(log)$
blacklist = runner\.(log)$

Can someone please help? what will be correct regex used to skip the logs with string called "runner"??

[manjunathmeti]

hi @sreesuresh545,

You just need to provide part of the file name i.e. _runner_.

[monitor:///project-abc/src/logs/*.log]
blacklist = \_runner\_

[kallinikos]

Thanks this worked for me. Finally, after lots of searching.

I blacklisted some Apache error logs as follows:

/var/log/httpd/X.XXXX.co.*-error_log

blacklist = \-error\_

[sreesuresh545]

Hi @manjunathmeti ,

Thanks for your response.

So, the "$" sign at the end of blacklist is not required?

I will try the mentioned option.

[manjunathmeti]

blacklist = <regular expression>

* If set, files from this input are NOT monitored if their path matches the specified regex.

* If a file matches the regexes in both the deny list and allow list settings, the file is NOT monitored. Deny lists take precedence over allow lists.

So it should be a regular expression that might contain $. $ is not mandatory.