Getting Data In

How to edit my inputs.conf to blacklist a directory?

Michael
Contributor

Apologies, but I'm not groking this. I've read dozens of "answers", I've read several docs on the topic. But, I can't find a way to blacklist a directory. Is it even possible? The Docs say it's possible, but demonstrate it with file extensions (ie.: ".bak$"), implying it can only be done on files, not directories... I have a deployer and universal clients, Splunk 6.5.

In this example, I'm simply trying to prevent gitlab entries. Please, can someone give me the straight answer as to what I should use? (Please don't give me a link to follow, chances are I've been there).

I'm using the stanza in my inputs.conf:

[monitor:///var/log]
disabled = 0

...and for the blacklist statement, I've tried a gazillion things, including:

blacklist = "/var/log/gitlab/*"
blacklist = \/var\/log\/gitlab\/*
blacklist = gitlab.$
blacklist = gitlab/.$
blacklist = ///var/log/gitlab
blacklist = %gitlab/%$
blacklist = gitlab\.$

and multiple variations of the above...
etc., etc., etc.

Thanks for your help,
~Frusterated

0 Karma

edoardo_vicendo
Contributor

I struggled a little bit mee too on those backlists in [monitor://] stanza.

The point is that you should write a regular expression and not the absolute path
Try this balcklist setting in your stanza, it should work

[monitor:///var/log]
disabled=0
blacklist=\/var\/log\/gitlab\/.*
0 Karma

dantimola
Communicator

This answer is late, but for this problem you can just add the stanza below for your inputs.conf

 [blacklist://<path>]

Cheers,
Dan

0 Karma

dwaddle
SplunkTrust
SplunkTrust

I will share a "clever trick" I use to decide how a whitelist / blacklist combination is going to work on a directory structure. Use the unix find command as follows:

find $PATH -print | egrep $WHITELIST | egrep -v $BLACKLIST

Things that survive this will be processed by the monitor stanza in question.

Michael
Contributor

Interesting...
will have to try this, thanks for sharing it!

0 Karma

somesoni2
Revered Legend

Try this

blacklist = \/gitlab\/
0 Karma

Michael
Contributor

Hey, it looks like it's working! blacklist = \/gitlab\/

BUT, that invites another mystery...

I've seen this before: I'll make changes in my deployment server, push the changes, and some will take effect immediately -- some won't until some time passes, like this one. This is the second time this has happened where I'll try something, it won't work, then I'll go home at the end of my workday (around 4:00 PM local). Then I'll come in the next day and it will be working. In both cases, I looked at the timeline and saw that the change kicked in (or, stopped getting log entries in this case) around 8:00 PM that same day.

Anyone else observe this behavior?

0 Karma

aaraneta_splunk
Splunk Employee
Splunk Employee

@Michael - Did the answer/comment provided by somesoni2 help provide a working solution to your original question? If yes, please let me know so I can convert it to an answer to be accepted. If no, please leave a comment with more feedback. Thanks!

0 Karma

lguinn2
Legend

In serverclass.conf, do you have the app set so that the forwarder automatically restarts? Because in general, the forwarder must restart in order for any changes to take effect. That includes changes to inputs.conf

0 Karma

Michael
Contributor

Yes.

As I said, most changes will take place within minutes. In fact, on this very system is question, when making changes, I actually made two: one pointed it to another index, and one was the blacklist. The index change took place immediately (confirming for me the app edit and reload was working as it should). But this odd delay in other things... happened twice now.

I wonder if it's a "bug" similar to what can happen when you manually edit the XML of a dashboard, and nothing you do with your browser refreshes the GUI until you execute a refresh...or wait.

Just an oddity, I guess it's not a big deal...

0 Karma

Michael
Contributor

Sorry, but nope. Has that actually worked for you, or are guessing?

I know my inputs.conf file is being deployed properly, because this system (Server Class) was previously feeding into another index. I used this to make other changes, including the index destination and "host = $decideOnStartup" (which is a very cool trick, BTW) -- and they all took. But I just can't get the "blacklist" to work...

0 Karma

somesoni2
Revered Legend

I haven't tested it but I've done similar thing for other use-cases of mine. Try being more specific and check the case of regex string.

blacklist = \/gitlab\/.+$

OR

blacklist = gitlab
0 Karma
Get Updates on the Splunk Community!

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...