Getting Data In

How does Splunk use wildcards for inputs in the backend?

neelamssantosh
Contributor

/var/log/…/apache.log matches the files in Splunk, but through either python or unix CLI, I am unable to perform the operation.

What/How exactly does Splunk function using wildcards in the backend?

Tags (2)
1 Solution

dwaddle
SplunkTrust
SplunkTrust

That is correct - the /.../ construct is not a part of standard *nix glob patterns. The best way to explain it is that when you use wildcards in an input stanza, splunk transmogrifies those into whitelist regexes. So for example:

[monitor:///var/log/.../apache.log]

will get transmogrified into something similar to:

[monitor:///var/log]
whitelist=^/var/log/(.*)/apache\.log$

and then this alternate-reality version is processed just like Splunk does any other monitor stanza with a whitelist. Similarly, * in a monitor stanza is transmogrified something like:

[monitor:///var/log/httpd/access*.log]

becomes

[monitor:///var/log/httpd]
whitelist=^/var/log/httpd/access[^/]*\.log$

These are probably not 100% exact representations of how the translation from glob-like-pattern to regex occurs but they are good examples of the concepts.

View solution in original post

dwaddle
SplunkTrust
SplunkTrust

That is correct - the /.../ construct is not a part of standard *nix glob patterns. The best way to explain it is that when you use wildcards in an input stanza, splunk transmogrifies those into whitelist regexes. So for example:

[monitor:///var/log/.../apache.log]

will get transmogrified into something similar to:

[monitor:///var/log]
whitelist=^/var/log/(.*)/apache\.log$

and then this alternate-reality version is processed just like Splunk does any other monitor stanza with a whitelist. Similarly, * in a monitor stanza is transmogrified something like:

[monitor:///var/log/httpd/access*.log]

becomes

[monitor:///var/log/httpd]
whitelist=^/var/log/httpd/access[^/]*\.log$

These are probably not 100% exact representations of how the translation from glob-like-pattern to regex occurs but they are good examples of the concepts.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...