I have an inputs.conf stanza that I want to add. I am adding it to monitor all files and sub-directories. Throughout these sub-directories there are .bz2 files that I dont want to ingest. What would be the best way? I was thinking blacklist the bz2 files but that may not be the best way? Maybe there's a better way?
[monitor:///var/log/remote/*]
blacklist=(*.bz2*)
index=nix_os
disabled = 0
There could be any number of sub-directories below the remote/ parent directory.
Thanks!
The asterisk *
will not recurse through subfolders. See this:
https://docs.splunk.com/Documentation/Splunk/8.0.1/Data/Specifyinputpathswithwildcards#Input_example...
I would use something like this. Assuming bz2 is the file extension
[monitor:///var/log/remote/*]
whitelist=\.bz2$
index=nix_os
disabled = 0
recursive=true #true by default
Splunk Docs has good documentation explaining how to whitelist and blacklist file paths. The blacklist
and whitelist
sections requires custom regex expressions. So yes, there are ways to blacklist all files of a specific file type .bz2 in your case.
See the Blacklist (ignore) files in Whitelist or blacklist specific incoming data
The asterisk *
will not recurse through subfolders. See this:
https://docs.splunk.com/Documentation/Splunk/8.0.1/Data/Specifyinputpathswithwildcards#Input_example...
I would use something like this. Assuming bz2 is the file extension
[monitor:///var/log/remote/*]
whitelist=\.bz2$
index=nix_os
disabled = 0
recursive=true #true by default