topic Re: Deploying blacklist configuration in inputs.conf to universal forwarder in Getting Data In

Deploying blacklist configuration in inputs.conf to universal forwarder

jcbrendsel — Sat, 14 Jan 2012 14:10:27 GMT

I am having problems blacklisting a sourcefile from being indexed.

We currently run version 4.3 and deploy configurations to a number of remote universal forwarders.

By default, our universal forwarder indexes index everything, defined as follows:

#/opt/splunkforwarder/etc/apps/search/local/inputs.conf
[monitor:///var/log]
disabled = false

There are several logs that are unnecessary and that generate large logs that I would like to stop from getting indexed. To do that, I modified inputs.conf on the indexer/search head as follows:

#/opt/splunk/etc/deployment-apps/forwarder/local/inputs.conf
[monitor:///var/log/httpd]
blacklist = goonhilly_access

As you can see, this inputs.conf file is different than the one that is used for the default configuration (search).

I then deployed this to the remote universal forwarder in question and restarted.

The problem is that the file is still getting indexed.

Do I have a problem with inputs.conf files being in conflict?

The
Any ideas?

Re: Deploying blacklist configuration in inputs.conf to universal forwarder

dwaddle — Sat, 14 Jan 2012 15:14:00 GMT

By default, monitor:// stanzas look for files recursively and settings in one stanza do not affect the other. So, yes, it's entirely possible that your /var/log/ stanza is ignoring your blacklist for /var/log/httpd. A couple of workable options include:

Blacklist "/var/log/httpd/.*" in your /var/log stanza and use the /var/log/httpd stanza to get all of those.
Blacklist goonhilly_access in your /var/log stanza and don't have a /var/log/httpd stanza at all.

There are CLI commands and REST endpoints to tell you which stanzas are detecting which files. One of the most useful ways of display this is with Amrit's script @ http://blogs.splunk.com/2011/01/02/did-i-miss-christmas-2/

Re: Deploying blacklist configuration in inputs.conf to universal forwarder

jcbrendsel — Sun, 15 Jan 2012 03:40:42 GMT

That did the trick. Thanks.

Re: Deploying blacklist configuration in inputs.conf to universal forwarder

Michael — Thu, 08 Oct 2015 15:00:00 GMT

I believe the line needs to be in regex format (according to: http://docs.splunk.com/Documentation/Splunk/6.3.0/Data/Whitelistorblacklistspecificincomingdata).

so, it should read:
goonhilly_access$

where $ = end of line