Getting Data In

Blacklisting clarification

Voltaire
Communicator

I am attempting to blacklist all files that end with these extensions in my inputs.conf file. The blacklist is not working correctly. These are image files located under a /images directory on the web server. These files are Not in the same directory as the log file. They are directory entries in the nginx-access log file contents.
For example

/wd/code/websites/wd-current/www/images/* 
/wd/code/websites/wd-current/www/js/resources/*
/wd/code/websites/sample.it/www/resources/*

I have attempted a few different methods. Any suggestions?

[default]
host = oh.br0ther.com

[monitor:///var/log/nginx-access.log]
blacklist= \.(jpg|png|gif|mov|js|swf|mp4|jar|signed|flv)$
Tags (1)
0 Karma

sdaniels
Splunk Employee
Splunk Employee

In your file monitoring stanza above you are referencing a file and not a directory. If I wanted to monitor the images directory and blacklist all of the image files I would so something like this:

[monitor:///wd/code/websites/wd-current/www/images/*]
index = myindex
sourcetype = mysourcetype
blacklist= \.(jpg|png|gif|mov|js|swf|mp4|jar|signed|flv)$

http://docs.splunk.com/Documentation/Splunk/5.0/Data/Monitorfilesanddirectories

0 Karma

Voltaire
Communicator

Nice ! thats where I found the reference. Thank you!

0 Karma

sdaniels
Splunk Employee
Splunk Employee

Some examples here. You can use elipsis wildcards or *.

http://docs.splunk.com/Documentation/Splunk/5.0/data/Specifyinputpathswithwildcards

0 Karma

Voltaire
Communicator

Excellent, just modified inputs.conf on my forwarder. Would this command work if I wanted to exclude all files under the www directory in any of the subfolders? or do I have to add a different syntax ?
for example
[monitor:///wd/code/websites/wd-current/www/.../*]

Thank you!

0 Karma

lguinn2
Legend

Your stanza will monitor only the following files:

1 - files named /var/log/nginx-access.log

2 - files underneath a directory named /var/log/nginx-access.log

Since this is not the stanza that is monitoring the directories that you name, putting the blacklist here will not help.

I don't see anything wrong with your blacklist, it just needs to be moved so that it will be part of the proper monitor stanza.

Voltaire
Communicator

Thank you Lisa!

0 Karma

Voltaire
Communicator

I am trying to block files from being read by splunk in those directories.
Thank you.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Are you trying to block files from being read by Splunk or to block specific lines of a logfile from being indexed?

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...