Deploying blacklist configuration in inputs.conf to universal forwarder

[jcbrendsel]

I am having problems blacklisting a sourcefile from being indexed.

We currently run version 4.3 and deploy configurations to a number of remote universal forwarders.

By default, our universal forwarder indexes index everything, defined as follows:

#/opt/splunkforwarder/etc/apps/search/local/inputs.conf
[monitor:///var/log]
disabled = false

There are several logs that are unnecessary and that generate large logs that I would like to stop from getting indexed. To do that, I modified inputs.conf on the indexer/search head as follows:

#/opt/splunk/etc/deployment-apps/forwarder/local/inputs.conf
[monitor:///var/log/httpd]
blacklist = goonhilly_access

As you can see, this inputs.conf file is different than the one that is used for the default configuration (search).

I then deployed this to the remote universal forwarder in question and restarted.

The problem is that the file is still getting indexed.

Do I have a problem with inputs.conf files being in conflict?

The
Any ideas?

[Michael]

I believe the line needs to be in regex format (according to: http://docs.splunk.com/Documentation/Splunk/6.3.0/Data/Whitelistorblacklistspecificincomingdata).

so, it should read:
goonhilly_access$

where $ = end of line

[dwaddle · Accepted solution]

By default, monitor:// stanzas look for files recursively and settings in one stanza do not affect the other. So, yes, it's entirely possible that your /var/log/ stanza is ignoring your blacklist for /var/log/httpd. A couple of workable options include:

1. Blacklist "/var/log/httpd/.*" in your /var/log stanza and use the /var/log/httpd stanza to get all of those.
2. Blacklist goonhilly_access in your /var/log stanza and don't have a /var/log/httpd stanza at all.

There are CLI commands and REST endpoints to tell you which stanzas are detecting which files. One of the most useful ways of display this is with Amrit's script @ http://blogs.splunk.com/2011/01/02/did-i-miss-christmas-2/

[jcbrendsel]

That did the trick. Thanks.