Getting Data In

overlapping directives in inputs.conf

mfrost8
Builder

I have a tree of files on a forwarder that looks something like the following:

/foo/able/
/foo/baker/
/foo/charlie/
/foo/delta/

I am currently monitoring them all (whitelisting some files under each directory to exclude ones that I don't want) but sending them all to the same index. Each of these directories represents a different application and I'd now like to look into putting some of them into separate indices. That is, I want the contents of /foo/b/* to go into a new index "b", but the rest of the directories would continue to go into "main" as they do now.

I thought the precendence in inputs.conf might be such that I could put more specific stanzas near the top, but this doesn't seem to work (i.e. everything goes into the main index):

[monitor:///foo/b*/]
_whitelist = .../(access|error)\.log$
index = baker

[monitor:///foo]
_whitelist = .../(access|error)\.log$
index = main

I'm wondering if my approach here is wrong. Is there a more appropriate way of accomplishing what I'm trying to do? There are too many directories (and we can add ones too often) to manually list all the directories under /foo in separate stanzas.

Thanks

1 Solution

piebob
Splunk Employee
Splunk Employee

i think the issue here is that specifying wildcards creates an implicit whitelist for that stanza. the longest fully qualified path is used as the monitor stanza, and the wildcards are translated into regular expressions.

the docs say "As a consequence, you can't have multiple stanzas with wildcards for files in the same directory. If you have multiple inputs that only disambiguate after a wildcard, they will collide."

check out more here:

http://www.splunk.com/base/Documentation/latest/Admin/Specifyinputpathswithwildcards#Wildcards_and_w...

View solution in original post

piebob
Splunk Employee
Splunk Employee

i think the issue here is that specifying wildcards creates an implicit whitelist for that stanza. the longest fully qualified path is used as the monitor stanza, and the wildcards are translated into regular expressions.

the docs say "As a consequence, you can't have multiple stanzas with wildcards for files in the same directory. If you have multiple inputs that only disambiguate after a wildcard, they will collide."

check out more here:

http://www.splunk.com/base/Documentation/latest/Admin/Specifyinputpathswithwildcards#Wildcards_and_w...

mfrost8
Builder

Yeah, after some experimentation, that seems to be the case. If I list out the directory names instead of wildcarding them, I can indeed get things to work as I'd like. Unfortunately, the "baker" directory on different machines takes slightly different forms ("baker4", "baker5" and so on). So it's looking like I'm going to have to do this the hard way -- without wildcards.

Thanks

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Accelerate Splunk App Development with AI

  Learn how to automate the repetitive parts of your build cycle.   Watch On-Demand   Replay the Accelerate ...

Splunk Agent Observability: Measuring AI Value with Splunk Tokenomics on AWS

The AI landscape is moving faster than ever, and our latest 2026 State of Observability report provides the ...

Enterprise Security Watch Party: What’s New in 8.8 - Recap

Splunk Community HQ hosted a Splunk Enterprise Security Watch Party: What’s New In ES 8.8 last Thursday. I had ...