Getting Data In

Trouble with global deny list in inputs.conf

JyPl4wNYu7GV1uL
Explorer

I've reviewed this issue: https://community.splunk.com/t5/Getting-Data-In/Can-I-edit-inputs-conf-to-initiate-a-global-blacklis...

And this page: https://docs.splunk.com/Documentation/Splunk/9.4.2/Data/Specifyinputpathswithwildcards

But I'm still having issues with a global deny list (eliminating any files with "*abcdef*" in the path) working.  It works fine when I put it in the [monitor] stanza but not globally when it's in the [default] stanza.  Syntax I've tried:

[default]
[blacklist:/var/.../*abcdef*/*]
[default]
[blacklist:///var/.../*abcdef*/*]
[default]
[blacklist:.*abcdef.*]

That last one I expected not to work, but I tried it anyway.  The first one looks correct to me.

This works fine, but I have to put it in many [monitor] stanzas (which seem silly).

[monitor:///var/.../mylog*.log]
blacklist = .*abcdef.*

Basically, I want to skip any files in the "/var" path where ".*abcdef.*" appears in the directory hierarchy. 

In most cases, I just have a "whitelist" and "blacklist" statement in each [monitor] stanza, but I have a case where I want to globally deny all files in a certain path.  I'm more comfortable with the regex syntax in the [monitor] stanza.  The global [blacklist:] syntax is not well documented.  I'm assuming it follows the same janky "sort of " regex rules like for the [monitor] stanza.

This is no help at all:

https://docs.splunk.com/Documentation/Splunk/9.4.2/Admin/Inputsconf

Because it doesn't well define <path>.

Labels (2)
0 Karma

JyPl4wNYu7GV1uL
Explorer

Hmm, based on that I thought I'd try fully specifying the full path.  Even this does not work:

[blacklist:/var/dir2/dir3/abcdef]

Where the full path is fully specified.  I still get files with that path from my [monitor] stanzas.  Giving up and sticking with the "blacklist = <regex>" in the monitor stanzas.

0 Karma

koronb_splunk
Splunk Employee
Splunk Employee

Hello! 😁

Have you tried [blacklist:///var/.*abcdef.*] opposed to [blacklist:/var/dir2/dir3/abcdef]




0 Karma

PickleRick
SplunkTrust
SplunkTrust

Unfortunately, the knowlegde around these parts of config is relatively sparse. You could test with one or two more slashes. And check your "list monitor" and "list inputstatus" outputs. Maybe they'll shed some light on this.

0 Karma

PickleRick
SplunkTrust
SplunkTrust

1. 

* The input treats a file as denied if the file starts with any of the
  defined deny list <paths>.

Never used the blacklist stanza but that suggests matching from the start and that's it. No fancy regexing or even wildcarding.

2. The [blacklist:...] stanza is _not_ an entry in default stanza. It's a separate stanza. [default] is meant for general settings which make sense in context of any input type. Blacklisting files isn't such setting (try blacklisting files in tcp: input). 

0 Karma
Get Updates on the Splunk Community!

Observe and Secure All Apps with Splunk

  Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

Splunk Decoded: Business Transactions vs Business IQ

It’s the morning of Black Friday, and your e-commerce site is handling 10x normal traffic. Orders are flowing, ...

Fastest way to demo Observability

I’ve been having a lot of fun learning about Kubernetes and Observability. I set myself an interesting ...