Getting Data In

overlapping directives in inputs.conf

mfrost8
Builder

I have a tree of files on a forwarder that looks something like the following:

/foo/able/
/foo/baker/
/foo/charlie/
/foo/delta/

I am currently monitoring them all (whitelisting some files under each directory to exclude ones that I don't want) but sending them all to the same index. Each of these directories represents a different application and I'd now like to look into putting some of them into separate indices. That is, I want the contents of /foo/b/* to go into a new index "b", but the rest of the directories would continue to go into "main" as they do now.

I thought the precendence in inputs.conf might be such that I could put more specific stanzas near the top, but this doesn't seem to work (i.e. everything goes into the main index):

[monitor:///foo/b*/]
_whitelist = .../(access|error)\.log$
index = baker

[monitor:///foo]
_whitelist = .../(access|error)\.log$
index = main

I'm wondering if my approach here is wrong. Is there a more appropriate way of accomplishing what I'm trying to do? There are too many directories (and we can add ones too often) to manually list all the directories under /foo in separate stanzas.

Thanks

1 Solution

piebob
Splunk Employee
Splunk Employee

i think the issue here is that specifying wildcards creates an implicit whitelist for that stanza. the longest fully qualified path is used as the monitor stanza, and the wildcards are translated into regular expressions.

the docs say "As a consequence, you can't have multiple stanzas with wildcards for files in the same directory. If you have multiple inputs that only disambiguate after a wildcard, they will collide."

check out more here:

http://www.splunk.com/base/Documentation/latest/Admin/Specifyinputpathswithwildcards#Wildcards_and_w...

View solution in original post

piebob
Splunk Employee
Splunk Employee

i think the issue here is that specifying wildcards creates an implicit whitelist for that stanza. the longest fully qualified path is used as the monitor stanza, and the wildcards are translated into regular expressions.

the docs say "As a consequence, you can't have multiple stanzas with wildcards for files in the same directory. If you have multiple inputs that only disambiguate after a wildcard, they will collide."

check out more here:

http://www.splunk.com/base/Documentation/latest/Admin/Specifyinputpathswithwildcards#Wildcards_and_w...

mfrost8
Builder

Yeah, after some experimentation, that seems to be the case. If I list out the directory names instead of wildcarding them, I can indeed get things to work as I'd like. Unfortunately, the "baker" directory on different machines takes slightly different forms ("baker4", "baker5" and so on). So it's looking like I'm going to have to do this the hard way -- without wildcards.

Thanks

0 Karma
Get Updates on the Splunk Community!

Earn a $35 Gift Card for Answering our Splunk Admins & App Developer Survey

Survey for Splunk Admins and App Developers is open now! | Earn a $35 gift card!      Hello there,  Splunk ...

Continuing Innovation & New Integrations Unlock Full Stack Observability For Your ...

You’ve probably heard the latest about AppDynamics joining the Splunk Observability portfolio, deepening our ...

Monitoring Amazon Elastic Kubernetes Service (EKS)

As we’ve seen, integrating Kubernetes environments with Splunk Observability Cloud is a quick and easy way to ...