topic Re: How to edit my inputs.conf to blacklist a directory? in Getting Data In

How to edit my inputs.conf to blacklist a directory?

Michael — Mon, 13 Feb 2017 19:20:24 GMT

Apologies, but I'm not groking this. I've read dozens of "answers", I've read several docs on the topic. But, I can't find a way to blacklist a directory. Is it even possible? The Docs say it's possible, but demonstrate it with file extensions (ie.: ".bak$"), implying it can only be done on files, not directories... I have a deployer and universal clients, Splunk 6.5.

In this example, I'm simply trying to prevent gitlab entries. Please, can someone give me the straight answer as to what I should use? (Please don't give me a link to follow, chances are I've been there).

I'm using the stanza in my inputs.conf:

[monitor:///var/log]
disabled = 0

...and for the blacklist statement, I've tried a gazillion things, including:

blacklist = "/var/log/gitlab/*"
blacklist = \/var\/log\/gitlab\/*
blacklist = gitlab.$
blacklist = gitlab/.$
blacklist = ///var/log/gitlab
blacklist = %gitlab/%$
blacklist = gitlab\.$

and multiple variations of the above...
etc., etc., etc.

Thanks for your help,
~Frusterated

Re: How to edit my inputs.conf to blacklist a directory?

somesoni2 — Mon, 13 Feb 2017 20:00:12 GMT

Try this

blacklist = \/gitlab\/

Re: How to edit my inputs.conf to blacklist a directory?

Michael — Mon, 13 Feb 2017 20:26:56 GMT

Sorry, but nope. Has that actually worked for you, or are guessing?

I know my inputs.conf file is being deployed properly, because this system (Server Class) was previously feeding into another index. I used this to make other changes, including the index destination and "host = $decideOnStartup" (which is a very cool trick, BTW) -- and they all took. But I just can't get the "blacklist" to work...

Re: How to edit my inputs.conf to blacklist a directory?

somesoni2 — Mon, 13 Feb 2017 20:51:16 GMT

I haven't tested it but I've done similar thing for other use-cases of mine. Try being more specific and check the case of regex string.

blacklist = \/gitlab\/.+$

blacklist = gitlab

Re: How to edit my inputs.conf to blacklist a directory?

Michael — Tue, 14 Feb 2017 21:20:58 GMT

Hey, it looks like it's working! blacklist = \/gitlab\/

BUT, that invites another mystery...

I've seen this before: I'll make changes in my deployment server, push the changes, and some will take effect immediately -- some won't until some time passes, like this one. This is the second time this has happened where I'll try something, it won't work, then I'll go home at the end of my workday (around 4:00 PM local). Then I'll come in the next day and it will be working. In both cases, I looked at the timeline and saw that the change kicked in (or, stopped getting log entries in this case) around 8:00 PM that same day.

Anyone else observe this behavior?

Re: How to edit my inputs.conf to blacklist a directory?

lguinn2 — Tue, 14 Feb 2017 23:25:24 GMT

In serverclass.conf, do you have the app set so that the forwarder automatically restarts? Because in general, the forwarder must restart in order for any changes to take effect. That includes changes to inputs.conf

Re: How to edit my inputs.conf to blacklist a directory?

dwaddle — Wed, 15 Feb 2017 04:06:35 GMT

I will share a "clever trick" I use to decide how a whitelist / blacklist combination is going to work on a directory structure. Use the unix find command as follows:

find $PATH -print | egrep $WHITELIST | egrep -v $BLACKLIST

Things that survive this will be processed by the monitor stanza in question.

Re: How to edit my inputs.conf to blacklist a directory?

Michael — Wed, 15 Feb 2017 13:11:14 GMT

Yes.

As I said, most changes will take place within minutes. In fact, on this very system is question, when making changes, I actually made two: one pointed it to another index, and one was the blacklist. The index change took place immediately (confirming for me the app edit and reload was working as it should). But this odd delay in other things... happened twice now.

I wonder if it's a "bug" similar to what can happen when you manually edit the XML of a dashboard, and nothing you do with your browser refreshes the GUI until you execute a refresh...or wait.

Just an oddity, I guess it's not a big deal...

Re: How to edit my inputs.conf to blacklist a directory?

Michael — Wed, 15 Feb 2017 13:12:08 GMT

Interesting...
will have to try this, thanks for sharing it!

Re: How to edit my inputs.conf to blacklist a directory?

aaraneta_splunk — Mon, 20 Mar 2017 00:33:14 GMT

@Michael - Did the answer/comment provided by somesoni2 help provide a working solution to your original question? If yes, please let me know so I can convert it to an answer to be accepted. If no, please leave a comment with more feedback. Thanks!

Re: How to edit my inputs.conf to blacklist a directory?

dantimola — Wed, 26 Jul 2017 04:33:14 GMT

This answer is late, but for this problem you can just add the stanza below for your inputs.conf

 [blacklist://<path>]

Cheers,
Dan

Re: How to edit my inputs.conf to blacklist a directory?

edoardo_vicendo — Thu, 30 Jan 2020 14:57:56 GMT

I struggled a little bit mee too on those backlists in [monitor://] stanza.

The point is that you should write a regular expression and not the absolute path
Try this balcklist setting in your stanza, it should work

[monitor:///var/log]
disabled=0
blacklist=\/var\/log\/gitlab\/.*

Re: How to edit my inputs.conf to blacklist a directory?

blbr123 — Wed, 25 Sep 2024 11:55:17 GMT

Hi @dwaddle ,

May I know where do we need to run this?

Can you please share some sample output?

Re: How to edit my inputs.conf to blacklist a directory?

isoutamo — Sat, 28 Sep 2024 15:20:45 GMT

Hi
as this is quite old thread, it's better to create a new question to get someone to answer you.
r. Ismo