All Apps and Add-ons

Splunk Add-on for Amazon Web Services 3.0: Why doesn't blacklist seem to be working for an S3 input?

muebel
SplunkTrust
SplunkTrust

I'm working with the Splunk Add-on for AWS 3.0, and am having an issue with the S3 input.

The S3 input has a blacklist config directive available. The bucket I'd like to input has binaries mixed in with the actual logs I am interested in, and so I configured the blacklist to exclude this type of file (along with .conf) by this regex:

(\.bin$|\.conf$)

However, the input is still indexing files with sources that end in .bin. Has anybody worked a similar issue? Is my understanding of the S3 input blacklist config incorrect?

The documentation for the input is here http://docs.splunk.com/Documentation/AddOns/latest/AWS/S3 , with the description for the blacklist config as:

A regular expression to indicate the S3 paths that the Splunk platform should exclude from scanning.

This seems fairly straightforward, and typical for a splunk blacklist, so it leaves me quite confused. Thanks for any help!

kchen_splunk
Splunk Employee
Splunk Employee

Please use the following regex for the blacklist. In short, the regex should be a exact match, not only containing.

.*(\.bin$|\.conf$)
0 Karma

muebel
SplunkTrust
SplunkTrust

that seems to make sense, but I ended up modifying the blacklist to be:
bin$
And it was effective at preventing the .bin inputs at least.

0 Karma

mreynov_splunk
Splunk Employee
Splunk Employee

if I am not mistaken, you may need to add astrerisk before, as the regex matches the entire path. So something like this:

 (*\.bin*|*\.conf*)
0 Karma