All Apps and Add-ons

Splunk Add-on for Amazon Web Services 3.0: Why doesn't blacklist seem to be working for an S3 input?

muebel
SplunkTrust
SplunkTrust

I'm working with the Splunk Add-on for AWS 3.0, and am having an issue with the S3 input.

The S3 input has a blacklist config directive available. The bucket I'd like to input has binaries mixed in with the actual logs I am interested in, and so I configured the blacklist to exclude this type of file (along with .conf) by this regex:

(\.bin$|\.conf$)

However, the input is still indexing files with sources that end in .bin. Has anybody worked a similar issue? Is my understanding of the S3 input blacklist config incorrect?

The documentation for the input is here http://docs.splunk.com/Documentation/AddOns/latest/AWS/S3 , with the description for the blacklist config as:

A regular expression to indicate the S3 paths that the Splunk platform should exclude from scanning.

This seems fairly straightforward, and typical for a splunk blacklist, so it leaves me quite confused. Thanks for any help!

kchen_splunk
Splunk Employee
Splunk Employee

Please use the following regex for the blacklist. In short, the regex should be a exact match, not only containing.

.*(\.bin$|\.conf$)
0 Karma

muebel
SplunkTrust
SplunkTrust

that seems to make sense, but I ended up modifying the blacklist to be:
bin$
And it was effective at preventing the .bin inputs at least.

0 Karma

mreynov_splunk
Splunk Employee
Splunk Employee

if I am not mistaken, you may need to add astrerisk before, as the regex matches the entire path. So something like this:

 (*\.bin*|*\.conf*)
0 Karma
Get Updates on the Splunk Community!

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...