Getting Data In

How to use wildcards and whitelists in monitor stanza (inputs.conf)?

plunkingalex
Explorer

I'm using a deployment server to distribute a single inputs.conf file to a number of servers in a class. The locations of the files that I need to monitor are similar between the servers, but sometimes (sub)directories refer to the servers instead of being generically named. This circumstance made me reach for wildcards / whitelists in determining the paths of the files to watch. (The alternative would be creating separate monitor stanzas for each individual server in the class, which defeats the purpose.) Can't get it to work, though. What am I missing?

These are the directories / files on the various servers I want to monitor:

/base/logs/appl/xxx.seg.ex/logfile1.log
/base/logs/appl/xxx.seg.ex/logfile2.log
/base/logs/appl/yyy.seg.ex/logfile1.log
/base/logs/appl/yyy.seg.ex/logfile2.log

And these are the monitor stanzas I'd set up in inputs.conf:

[monitor:///base/logs/appl/*.seg.ex/logfile1.log]
index=index

[monitor:///base/logs/appl/*.seg.ex/logfile2.log]
index=index

Unfortunately this does not work...

Checking the _internal index made clear that the monitor stanzas are not OK. Apparently implicit whitelists were added:

'^\/base\/logs\/appl/[^/]*.seg.ex/logfile1.log$' (on path 'monitor:///base/logs/appl') [1]

'^\/base\/logs\/appl/[^/]*.seg.ex/logfile2.log$' (on path 'monitor:///base/logs/appl') [2]

The _internal index also contains logevents saying:

TailingProcessor - Will not call watch on path '/base/logs/appl/xxx.seg.ex/logfile1.log due to stanza: monitor:///base/logs/appl/*.seg.ex/logfile1.log [1]

TailingProcessor - Will not call watch on path '/base/logs/appl/xxx.seg.ex/logfile2.log due to stanza: monitor:///base/logs/appl/*.seg.ex/logfile1.log [2]

Why doesn't this work? And how could I get it to work as desired?

0 Karma

codebuilder
SplunkTrust
SplunkTrust

The problem is that because of the wildcard, you are telling the forwarder to essentially monitor the same directory and files, but send them to two different indexes.

You'll need to further delimit the directory or file names being monitored so that they are unique.
Splunk can send the same file to two different indexes, but not using the configuration that you have in place.

----
An upvote would be appreciated and Accept Solution if it helps!

plunkingalex
Explorer

Will try to set-up an instance where I can test; want to see if I can replicate this behaviour and rule out a specific issue with this environment.

0 Karma

codebuilder
SplunkTrust
SplunkTrust

I'm not sure what your filesystem structure looks like, but it could be a recursive issue.

Try using this:
[monitor:///base/logs/appl/.../logfile1.log]

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

plunkingalex
Explorer

Tried inputs.conf with only one monitor stanza, which I also simplified by replacing a whole segment of the path with an * (instead using the wildcard for a part of a segment):

[monitor:///base/logs/appl/*/logfile1.log]
index=index

This configuration is almost exactly the same as one shown in the examples here: https://docs.splunk.com/Documentation/Splunk/8.0.3/Data/Specifyinputpathswithwildcards

Strangely, it still doen't work. The same / similar errors in _internal.

0 Karma

codebuilder
SplunkTrust
SplunkTrust

Did you cycle Splunk after making the changes?

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

plunkingalex
Explorer

If by 'cycle', you mean 'restart the forwarder' then yes. I performed a restart of the forwarder every time I changed the inputs.config file.

0 Karma

plunkingalex
Explorer

Tried inputs.conf without the second monitor stanza, so there was only:

[monitor:///base/logs/appl/*.seg.ex/logfile1.log]
index=index

Didn't work; the same / similar events pop up in the _internal index.

0 Karma

plunkingalex
Explorer

Thanks for your reply, codebuilder. Changed the index stanza so that both files go to the same index, but I still get the same errors. What gives?!

0 Karma
*NEW* Splunk Love Promo!
Snag a $25 Visa Gift Card for Giving Your Review!

It's another Splunk Love Special! For a limited time, you can review one of our select Splunk products through Gartner Peer Insights and receive a $25 Visa gift card!

Review:





Or Learn More in Our Blog >>