Getting Data In

overlapping directives in inputs.conf

mfrost8
Builder

I have a tree of files on a forwarder that looks something like the following:

/foo/able/
/foo/baker/
/foo/charlie/
/foo/delta/

I am currently monitoring them all (whitelisting some files under each directory to exclude ones that I don't want) but sending them all to the same index. Each of these directories represents a different application and I'd now like to look into putting some of them into separate indices. That is, I want the contents of /foo/b/* to go into a new index "b", but the rest of the directories would continue to go into "main" as they do now.

I thought the precendence in inputs.conf might be such that I could put more specific stanzas near the top, but this doesn't seem to work (i.e. everything goes into the main index):

[monitor:///foo/b*/]
_whitelist = .../(access|error)\.log$
index = baker

[monitor:///foo]
_whitelist = .../(access|error)\.log$
index = main

I'm wondering if my approach here is wrong. Is there a more appropriate way of accomplishing what I'm trying to do? There are too many directories (and we can add ones too often) to manually list all the directories under /foo in separate stanzas.

Thanks

1 Solution

piebob
Splunk Employee
Splunk Employee

i think the issue here is that specifying wildcards creates an implicit whitelist for that stanza. the longest fully qualified path is used as the monitor stanza, and the wildcards are translated into regular expressions.

the docs say "As a consequence, you can't have multiple stanzas with wildcards for files in the same directory. If you have multiple inputs that only disambiguate after a wildcard, they will collide."

check out more here:

http://www.splunk.com/base/Documentation/latest/Admin/Specifyinputpathswithwildcards#Wildcards_and_w...

View solution in original post

piebob
Splunk Employee
Splunk Employee

i think the issue here is that specifying wildcards creates an implicit whitelist for that stanza. the longest fully qualified path is used as the monitor stanza, and the wildcards are translated into regular expressions.

the docs say "As a consequence, you can't have multiple stanzas with wildcards for files in the same directory. If you have multiple inputs that only disambiguate after a wildcard, they will collide."

check out more here:

http://www.splunk.com/base/Documentation/latest/Admin/Specifyinputpathswithwildcards#Wildcards_and_w...

mfrost8
Builder

Yeah, after some experimentation, that seems to be the case. If I list out the directory names instead of wildcarding them, I can indeed get things to work as I'd like. Unfortunately, the "baker" directory on different machines takes slightly different forms ("baker4", "baker5" and so on). So it's looking like I'm going to have to do this the hard way -- without wildcards.

Thanks

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...