Getting Data In

How to whitelist files in directory and not in subdirectories?

akchauhan
Explorer

Hi Guys
I am trying to pick logs having job-info.*log name in common directory and job-heartbeat.*logs from heartbeat sub directory and job-error .*log from error sub directory. I used the configuration below and it is working fine. The only issue that I am facing is job-info.*log files are also getting picked up when someone places these files in other sub directories (xyz in below case). I don't want these files to pick when they are placed in any sub-directories, These should be picked only when they are placed in the common directory. Please suggest changes in the whitelist.

[monitor:///abc/common/]
disabled = false
index = infra_job
whitelist = (job-info.*log|heartbeat/job-heartbeat.*log|error/job-error.*log)

directories structure

cd /abc/common/

files/directories under this directory (example)

error/
heartbeat/
xyz/job-info.*log ---- don't want these logs to pick
job-info1.log
job-info2.log
job-info3.log

Thanks in advance

0 Karma

FrankVl
Ultra Champion

Why not create 3 separate monitor inputs, one for job-info, one for error and one for heartbeat?

[monitor:///abc/common/job-info.*log]
disabled = false
index = infra_job

[monitor:///abc/common/heartbeat/job-heartbeat.*log]
disabled = false
index = infra_job

[monitor:///abc/common/error/job-error.*log]
disabled = false
index = infra_job
0 Karma

akchauhan
Explorer

yes, this was already in mind. Just thinking instead of creating 3 separate model inputs, if it is possible to do using whitelist/blacklist in single input that would be great.

0 Karma

horsefez
Motivator

Hey akchauhan,

there is a blacklist setting you could set for the job-info*.log files.

Just add the following line to your config:

blacklist=(\/abc\/common\/xyz\/job-info\d*\.log)

This will make sure you definitely will not index files under the specified path, cuz blacklist take precedence over whitelists.
Stated here: If a file matches the regexes in both the blacklist and whitelist settings,
the file is NOT monitored. Blacklists take precedence over whitelists.

0 Karma

akchauhan
Explorer

There are chances that dev team can create directory similar to xyz with other name as well. so what change do I need to add in blacklist you mentioned?

0 Karma

ddrillic
Ultra Champion
recursive = false

might do it.

0 Karma

akchauhan
Explorer

this is not working. Adding recursive = false will not pick logs in error and heartbeat directory.

0 Karma
Get Updates on the Splunk Community!

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...

Brains, Bytes, and Boston: Learn from the Best at .conf25

When you think of Boston, you might picture colonial charm, world-class universities, or even the crack of a ...