Getting Data In

How does Splunk use wildcards for inputs in the backend?

neelamssantosh
Contributor

/var/log/…/apache.log matches the files in Splunk, but through either python or unix CLI, I am unable to perform the operation.

What/How exactly does Splunk function using wildcards in the backend?

Tags (2)
1 Solution

dwaddle
SplunkTrust
SplunkTrust

That is correct - the /.../ construct is not a part of standard *nix glob patterns. The best way to explain it is that when you use wildcards in an input stanza, splunk transmogrifies those into whitelist regexes. So for example:

[monitor:///var/log/.../apache.log]

will get transmogrified into something similar to:

[monitor:///var/log]
whitelist=^/var/log/(.*)/apache\.log$

and then this alternate-reality version is processed just like Splunk does any other monitor stanza with a whitelist. Similarly, * in a monitor stanza is transmogrified something like:

[monitor:///var/log/httpd/access*.log]

becomes

[monitor:///var/log/httpd]
whitelist=^/var/log/httpd/access[^/]*\.log$

These are probably not 100% exact representations of how the translation from glob-like-pattern to regex occurs but they are good examples of the concepts.

View solution in original post

dwaddle
SplunkTrust
SplunkTrust

That is correct - the /.../ construct is not a part of standard *nix glob patterns. The best way to explain it is that when you use wildcards in an input stanza, splunk transmogrifies those into whitelist regexes. So for example:

[monitor:///var/log/.../apache.log]

will get transmogrified into something similar to:

[monitor:///var/log]
whitelist=^/var/log/(.*)/apache\.log$

and then this alternate-reality version is processed just like Splunk does any other monitor stanza with a whitelist. Similarly, * in a monitor stanza is transmogrified something like:

[monitor:///var/log/httpd/access*.log]

becomes

[monitor:///var/log/httpd]
whitelist=^/var/log/httpd/access[^/]*\.log$

These are probably not 100% exact representations of how the translation from glob-like-pattern to regex occurs but they are good examples of the concepts.

Get Updates on the Splunk Community!

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...

Splunkbase | Splunk Dashboard Examples App for SimpleXML End of Life

The Splunk Dashboard Examples App for SimpleXML will reach end of support on Dec 19, 2024, after which no new ...