Getting Data In

Deploying blacklist configuration in inputs.conf to universal forwarder

jcbrendsel
Path Finder

I am having problems blacklisting a sourcefile from being indexed.

We currently run version 4.3 and deploy configurations to a number of remote universal forwarders.

By default, our universal forwarder indexes index everything, defined as follows:

#/opt/splunkforwarder/etc/apps/search/local/inputs.conf
[monitor:///var/log]
disabled = false

There are several logs that are unnecessary and that generate large logs that I would like to stop from getting indexed. To do that, I modified inputs.conf on the indexer/search head as follows:

#/opt/splunk/etc/deployment-apps/forwarder/local/inputs.conf
[monitor:///var/log/httpd]
blacklist = goonhilly_access

As you can see, this inputs.conf file is different than the one that is used for the default configuration (search).

I then deployed this to the remote universal forwarder in question and restarted.

The problem is that the file is still getting indexed.

Do I have a problem with inputs.conf files being in conflict?

The
Any ideas?

0 Karma
1 Solution

dwaddle
SplunkTrust
SplunkTrust

By default, monitor:// stanzas look for files recursively and settings in one stanza do not affect the other. So, yes, it's entirely possible that your /var/log/ stanza is ignoring your blacklist for /var/log/httpd. A couple of workable options include:

  1. Blacklist "/var/log/httpd/.*" in your /var/log stanza and use the /var/log/httpd stanza to get all of those.
  2. Blacklist goonhilly_access in your /var/log stanza and don't have a /var/log/httpd stanza at all.

There are CLI commands and REST endpoints to tell you which stanzas are detecting which files. One of the most useful ways of display this is with Amrit's script @ http://blogs.splunk.com/2011/01/02/did-i-miss-christmas-2/

View solution in original post

Michael
Contributor

I believe the line needs to be in regex format (according to: http://docs.splunk.com/Documentation/Splunk/6.3.0/Data/Whitelistorblacklistspecificincomingdata).

so, it should read:
goonhilly_access$

where $ = end of line

0 Karma

dwaddle
SplunkTrust
SplunkTrust

By default, monitor:// stanzas look for files recursively and settings in one stanza do not affect the other. So, yes, it's entirely possible that your /var/log/ stanza is ignoring your blacklist for /var/log/httpd. A couple of workable options include:

  1. Blacklist "/var/log/httpd/.*" in your /var/log stanza and use the /var/log/httpd stanza to get all of those.
  2. Blacklist goonhilly_access in your /var/log stanza and don't have a /var/log/httpd stanza at all.

There are CLI commands and REST endpoints to tell you which stanzas are detecting which files. One of the most useful ways of display this is with Amrit's script @ http://blogs.splunk.com/2011/01/02/did-i-miss-christmas-2/

jcbrendsel
Path Finder

That did the trick. Thanks.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...