Getting Data In

How to whitelist files in directory and not in subdirectories?

akchauhan
Explorer

Hi Guys
I am trying to pick logs having job-info.*log name in common directory and job-heartbeat.*logs from heartbeat sub directory and job-error .*log from error sub directory. I used the configuration below and it is working fine. The only issue that I am facing is job-info.*log files are also getting picked up when someone places these files in other sub directories (xyz in below case). I don't want these files to pick when they are placed in any sub-directories, These should be picked only when they are placed in the common directory. Please suggest changes in the whitelist.

[monitor:///abc/common/]
disabled = false
index = infra_job
whitelist = (job-info.*log|heartbeat/job-heartbeat.*log|error/job-error.*log)

directories structure

cd /abc/common/

files/directories under this directory (example)

error/
heartbeat/
xyz/job-info.*log ---- don't want these logs to pick
job-info1.log
job-info2.log
job-info3.log

Thanks in advance

0 Karma

FrankVl
Ultra Champion

Why not create 3 separate monitor inputs, one for job-info, one for error and one for heartbeat?

[monitor:///abc/common/job-info.*log]
disabled = false
index = infra_job

[monitor:///abc/common/heartbeat/job-heartbeat.*log]
disabled = false
index = infra_job

[monitor:///abc/common/error/job-error.*log]
disabled = false
index = infra_job
0 Karma

akchauhan
Explorer

yes, this was already in mind. Just thinking instead of creating 3 separate model inputs, if it is possible to do using whitelist/blacklist in single input that would be great.

0 Karma

horsefez
SplunkTrust
SplunkTrust

Hey akchauhan,

there is a blacklist setting you could set for the job-info*.log files.

Just add the following line to your config:

blacklist=(\/abc\/common\/xyz\/job-info\d*\.log)

This will make sure you definitely will not index files under the specified path, cuz blacklist take precedence over whitelists.
Stated here: If a file matches the regexes in both the blacklist and whitelist settings,
the file is NOT monitored. Blacklists take precedence over whitelists.

0 Karma

akchauhan
Explorer

There are chances that dev team can create directory similar to xyz with other name as well. so what change do I need to add in blacklist you mentioned?

0 Karma

ddrillic
Ultra Champion
recursive = false

might do it.

0 Karma

akchauhan
Explorer

this is not working. Adding recursive = false will not pick logs in error and heartbeat directory.

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...