Getting Data In

overlapping directives in inputs.conf

Builder

I have a tree of files on a forwarder that looks something like the following:

/foo/able/
/foo/baker/
/foo/charlie/
/foo/delta/

I am currently monitoring them all (whitelisting some files under each directory to exclude ones that I don't want) but sending them all to the same index. Each of these directories represents a different application and I'd now like to look into putting some of them into separate indices. That is, I want the contents of /foo/b/* to go into a new index "b", but the rest of the directories would continue to go into "main" as they do now.

I thought the precendence in inputs.conf might be such that I could put more specific stanzas near the top, but this doesn't seem to work (i.e. everything goes into the main index):

[monitor:///foo/b*/]
_whitelist = .../(access|error)\.log$
index = baker

[monitor:///foo]
_whitelist = .../(access|error)\.log$
index = main

I'm wondering if my approach here is wrong. Is there a more appropriate way of accomplishing what I'm trying to do? There are too many directories (and we can add ones too often) to manually list all the directories under /foo in separate stanzas.

Thanks

1 Solution

Motivator

i think the issue here is that specifying wildcards creates an implicit whitelist for that stanza. the longest fully qualified path is used as the monitor stanza, and the wildcards are translated into regular expressions.

the docs say "As a consequence, you can't have multiple stanzas with wildcards for files in the same directory. If you have multiple inputs that only disambiguate after a wildcard, they will collide."

check out more here:

http://www.splunk.com/base/Documentation/latest/Admin/Specifyinputpathswithwildcards#Wildcards_and_w...

View solution in original post

Motivator

i think the issue here is that specifying wildcards creates an implicit whitelist for that stanza. the longest fully qualified path is used as the monitor stanza, and the wildcards are translated into regular expressions.

the docs say "As a consequence, you can't have multiple stanzas with wildcards for files in the same directory. If you have multiple inputs that only disambiguate after a wildcard, they will collide."

check out more here:

http://www.splunk.com/base/Documentation/latest/Admin/Specifyinputpathswithwildcards#Wildcards_and_w...

View solution in original post

Builder

Yeah, after some experimentation, that seems to be the case. If I list out the directory names instead of wildcarding them, I can indeed get things to work as I'd like. Unfortunately, the "baker" directory on different machines takes slightly different forms ("baker4", "baker5" and so on). So it's looking like I'm going to have to do this the hard way -- without wildcards.

Thanks

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!