How to edit my inputs.conf to blacklist a directory?

[Michael]

Apologies, but I'm not groking this. I've read dozens of "answers", I've read several docs on the topic. But, I can't find a way to blacklist a directory. Is it even possible? The Docs say it's possible, but demonstrate it with file extensions (ie.: ".bak$"), implying it can only be done on files, not directories... I have a deployer and universal clients, Splunk 6.5.

In this example, I'm simply trying to prevent gitlab entries. Please, can someone give me the straight answer as to what I should use? (Please don't give me a link to follow, chances are I've been there).

I'm using the stanza in my inputs.conf:

[monitor:///var/log]
disabled = 0

...and for the blacklist statement, I've tried a gazillion things, including:

blacklist = "/var/log/gitlab/*"
blacklist = \/var\/log\/gitlab\/*
blacklist = gitlab.$
blacklist = gitlab/.$
blacklist = ///var/log/gitlab
blacklist = %gitlab/%$
blacklist = gitlab\.$

and multiple variations of the above...
etc., etc., etc.

Thanks for your help,
~Frusterated

[edoardo_vicendo]

I struggled a little bit mee too on those backlists in [monitor://] stanza.

The point is that you should write a regular expression and not the absolute path
Try this balcklist setting in your stanza, it should work

[monitor:///var/log]
disabled=0
blacklist=\/var\/log\/gitlab\/.*

[dantimola]

This answer is late, but for this problem you can just add the stanza below for your inputs.conf

 [blacklist://<path>]

Cheers,
Dan

[dwaddle]

I will share a "clever trick" I use to decide how a whitelist / blacklist combination is going to work on a directory structure. Use the unix find command as follows:

find $PATH -print | egrep $WHITELIST | egrep -v $BLACKLIST

Things that survive this will be processed by the monitor stanza in question.

[Michael]

Interesting...
will have to try this, thanks for sharing it!

[somesoni2]

Try this

blacklist = \/gitlab\/

[Michael]

Hey, it looks like it's working! blacklist = \/gitlab\/

BUT, that invites another mystery...

I've seen this before: I'll make changes in my deployment server, push the changes, and some will take effect immediately -- some won't until some time passes, like this one. This is the second time this has happened where I'll try something, it won't work, then I'll go home at the end of my workday (around 4:00 PM local). Then I'll come in the next day and it will be working. In both cases, I looked at the timeline and saw that the change kicked in (or, stopped getting log entries in this case) around 8:00 PM that same day.

Anyone else observe this behavior?

[aaraneta_splunk]

@Michael - Did the answer/comment provided by somesoni2 help provide a working solution to your original question? If yes, please let me know so I can convert it to an answer to be accepted. If no, please leave a comment with more feedback. Thanks!

[lguinn2]

In serverclass.conf, do you have the app set so that the forwarder automatically restarts? Because in general, the forwarder must restart in order for any changes to take effect. That includes changes to inputs.conf

[Michael]

Yes.

As I said, most changes will take place within minutes. In fact, on this very system is question, when making changes, I actually made two: one pointed it to another index, and one was the blacklist. The index change took place immediately (confirming for me the app edit and reload was working as it should). But this odd delay in other things... happened twice now.

I wonder if it's a "bug" similar to what can happen when you manually edit the XML of a dashboard, and nothing you do with your browser refreshes the GUI until you execute a refresh...or wait.

Just an oddity, I guess it's not a big deal...

[Michael]

Sorry, but nope. Has that actually worked for you, or are guessing?

I know my inputs.conf file is being deployed properly, because this system (Server Class) was previously feeding into another index. I used this to make other changes, including the index destination and "host = $decideOnStartup" (which is a very cool trick, BTW) -- and they all took. But I just can't get the "blacklist" to work...

[somesoni2]

I haven't tested it but I've done similar thing for other use-cases of mine. Try being more specific and check the case of regex string.

blacklist = \/gitlab\/.+$

OR

blacklist = gitlab