overlapping directives in inputs.conf

[mfrost8]

I have a tree of files on a forwarder that looks something like the following:

/foo/able/
/foo/baker/
/foo/charlie/
/foo/delta/

I am currently monitoring them all (whitelisting some files under each directory to exclude ones that I don't want) but sending them all to the same index. Each of these directories represents a different application and I'd now like to look into putting some of them into separate indices. That is, I want the contents of /foo/b/* to go into a new index "b", but the rest of the directories would continue to go into "main" as they do now.

I thought the precendence in inputs.conf might be such that I could put more specific stanzas near the top, but this doesn't seem to work (i.e. everything goes into the main index):

[monitor:///foo/b*/]
_whitelist = .../(access|error)\.log$
index = baker

[monitor:///foo]
_whitelist = .../(access|error)\.log$
index = main

I'm wondering if my approach here is wrong. Is there a more appropriate way of accomplishing what I'm trying to do? There are too many directories (and we can add ones too often) to manually list all the directories under /foo in separate stanzas.

Thanks

[piebob · Accepted solution]

i think the issue here is that specifying wildcards creates an implicit whitelist for that stanza. the longest fully qualified path is used as the monitor stanza, and the wildcards are translated into regular expressions.

the docs say "As a consequence, you can't have multiple stanzas with wildcards for files in the same directory. If you have multiple inputs that only disambiguate after a wildcard, they will collide."

check out more here:

http://www.splunk.com/base/Documentation/latest/Admin/Specifyinputpathswithwildcards#Wildcards_and_w...

[mfrost8]

Yeah, after some experimentation, that seems to be the case. If I list out the directory names instead of wildcarding them, I can indeed get things to work as I'd like. Unfortunately, the "baker" directory on different machines takes slightly different forms ("baker4", "baker5" and so on). So it's looking like I'm going to have to do this the hard way -- without wildcards.

Thanks