Getting Data In
Highlighted

Can we configure Splunk to not look inside archive files?

New Member

Hello,

By default:
Splunk Enterprise decompresses archive files before it indexes them. It can handle these common archive file types: tar, gz, bz2, tar.gz, tgz, tbz, tbz2, zip, and z.
(http://docs.splunk.com/Documentation/Splunk/6.1.6/Data/Monitorfilesanddirectories)

Is it possible to configure Splunk to not do this? Or another way to handle our scenario?

We have a Windows directory input path that we are indexing *.log files. The problem is, there are .zip files in that folder that also contain *.log files, but we want to ignore those.

Thanks in advance.

0 Karma
Highlighted

Re: Can we configure Splunk to not look inside archive files?

Ultra Champion

All that you need to do is to specify in the monitor "just" the *.log files.

View solution in original post

Highlighted

Re: Can we configure Splunk to not look inside archive files?

Splunk Employee
Splunk Employee

Assuming your log files exists in C:\Logs\ or sub directories.

- inputs.conf
[monitor://\C:\Logs\....log]

Or, you can make use of white list

- inputs.conf
[monitor://\C:\Logs]
whitelist = \.log$
Highlighted

Re: Can we configure Splunk to not look inside archive files?

New Member

Masa ; ddrillic,

Thanks for your replies, however, this does not seem to work. Splunk is still looking within the zip file and finding the .log files within it.

It seems as if it is decompressing the archive and finding the .log files within it. I believe it is the decompression that we need to avoid.

0 Karma
Highlighted

Re: Can we configure Splunk to not look inside archive files?

Ultra Champion

What I normally do is being very explicit to the level of the files and not just the directory. Something like - [monitor://\C:\Logs\location\log\*.log]

Using this variation ensures that only files with extension of .log will be processed.

0 Karma
Highlighted

Re: Can we configure Splunk to not look inside archive files?

Splunk Employee
Splunk Employee

Have you restarted Splunk?

F.Y.I.

[monitor://\C:\Logs\location\log\*.log]

Splunk will translated this stanza to;

[monitor://\C:\Logs\location\log]
whiltelist = [^\//]+\.log
0 Karma
Highlighted

Re: Can we configure Splunk to not look inside archive files?

Splunk Employee
Splunk Employee

Also, can you send us example of a file path and the configuration you used?

0 Karma
Highlighted

Re: Can we configure Splunk to not look inside archive files?

Splunk Employee
Splunk Employee

I agree with ddrllic.

0 Karma
Highlighted

Re: Can we configure Splunk to not look inside archive files?

New Member

Thank you both for your help.

Using the whitelist does look like it works. We were getting confused by the number of files that appear in the Files and Directory input for that folder. That number seems to represent the number of files found (plus the root folder), not necessarily the ones it has indexed.

0 Karma