inputs.conf whitelist blacklist question

[ebailey]

Greetings

I have trying to gather logs by sifting through three levels of the file system with a white list and blacklist. It is not working and outside of creating a very long list of monitors I am not sure what to do anymore.

Any help is much appreciated.

Thanks!

My config

[monitor:///opt/transient/prod/data/xxxxxx/xxx/xx_ci/log/]
recursive = true
sourcetype=prd_xxx_xx_log
disabled = false
crcSalt =
whitelist = .dat$|.log$|.err$|.txt$
blacklist = .gz$

[monitor:///opt/transient/prod/data/xxxxxx/xxx/xxx_ci/error/]
recursive = true
sourcetype=prd_xxx_xx_error
disabled = false
crcSalt =
whitelist = .dat$|.log$|.err$|.txt$
blacklist = .gz$

Any ideas?

[neelamssantosh]

why to give 'blacklist' of Specific extensions of compressed files to exclude, where splunk already ignores..

packed_extensions_list:
bz, bz2, tbz, tbz2, Z, gz, tgz, tar, zip

[MuS]

hmmm, this packed_extensions_list is an option to crawl.conf only http://docs.splunk.com/Documentation/Splunk/6.1.3/admin/Crawlconf

and here would be the correct statement regarding compressed files from http://docs.splunk.com/Documentation/Splunk/6.1.3/Data/MonitorFilesandDirectories

• Splunk Enterprise decompresses archive files before it indexes them. It can handle these common archive file types: tar, gz, bz2, tar.gz, tgz, tbz, bz2, zip, and z.

[lguinn2 · Accepted solution]

Yes! Ditch the blacklist. You only need the whitelist. You also don't need the recursive - that's the default.

Finally - why do you have the crcSalt?

[ebailey]

Thanks!

I use crcSalt with the expectation it is going to be needed. I pulled it out.

Question - I made the changes you recommended and the results look good except now I am picking up files such as

test.dat.lock also I am picking up all . files such as

.work

Any ideas?

Thanks

Ed