Getting Data In

Blacklist directory names not working

jackal242
Engager

I've added the following blacklist line:

[monitor:///usr/local/alert/logs]
blacklist = (bak|sqlsync|syncdb_log|sql_bak|WebCorder)

And yet I see in my sources the following files are being indexed for that monitor:

/usr/local/alert/logs/bak/ipdlog_new.12133 | 17,016
/usr/local/alert/logs/bak/ipdlog_new.18100 | 15,770
/usr/local/alert/logs/bak/ipdlog_new.9881 | 15,727 
/usr/local/alert/logs/WebCorder/FF27774.65391.72094_001_act_src.html | 1
/usr/local/alert/logs/WebCorder/FF27774.65391.72094_002_act_src.html | 1

So clearly the blacklist is not working.

Any help?

0 Karma
1 Solution

jackal242
Engager

I found the problem.

The problem is that I had already indexed the directory once without the blacklist and nothing removed those sources.

I thought removing the monitor and readding it with the blacklist would remove all the unwanted directories.

I had to do a "splunk clean eventdata" to get it to remove all the unwanted source directories that are now in the blacklist.

View solution in original post

dwaddle
SplunkTrust
SplunkTrust

(Some additional information for you)

The "Search" app's initial Dashboard (the Summary page) is generated by running several searches against the already-indexed data. The sources list runs the following search:

| metadata sources

This includes an up to date event count which is updated roughly every 25 seconds. More info on the metadata command is available at http://www.splunk.com/base/Documentation/latest/SearchReference/Metadata.

As you commented, removing an input from Splunk does not delete already indexed data from the index - it stops Splunk from indexing any additional data from those files.

dwaddle
SplunkTrust
SplunkTrust

Thanks Amrit for the additional detail here.

0 Karma

jackal242
Engager

I found the problem.

The problem is that I had already indexed the directory once without the blacklist and nothing removed those sources.

I thought removing the monitor and readding it with the blacklist would remove all the unwanted directories.

I had to do a "splunk clean eventdata" to get it to remove all the unwanted source directories that are now in the blacklist.

View solution in original post