topic Re: How can I use wildcards (*) for the source stanza in props.conf? in Getting Data In

How can I use wildcards (*) for the source stanza in props.conf?

jonatanjosefson — Tue, 27 Oct 2015 16:15:33 GMT

Hi,

In my live splunk environment, I have a syslog receiver on a Linux machine putting all incoming logs in /opt/splunk/var/log/syslog/
In the syslog folder, sub folders are created for each syslog source and in that sub folder, each source's log files are written down.
Example: /opt/splunk/var/log/syslog/example123-lx0001-10.10.10.10/example.log
I then pull those log files into Splunk via an agent/UF.

That syslog receiver (that's actually several machines) receives logs from several hundreds of hosts and what I want to do is to apply a props configuration on every source that includes lx001 in the host name of the syslog source.

What I've tried, but not gotten to work:

[source::/opt/splunk/var/log/syslog/*lx0001*]

I would be so glad if someone could lead me in the right direction.

Re: How can I use wildcards (*) for the source stanza in props.conf?

somesoni2 — Tue, 27 Oct 2015 16:47:45 GMT

How about this

[source::/opt/splunk/var/log/syslog/*lx0001*]

Re: How can I use wildcards (*) for the source stanza in props.conf?

jonatanjosefson — Wed, 28 Oct 2015 05:51:38 GMT

I corrected my post since it in fact was possible to write wildcards in the text. Unfortunately that stanza is what I've tried and not got to work.

Re: How can I use wildcards (*) for the source stanza in props.conf?

lguinn2 — Wed, 28 Oct 2015 06:35:00 GMT

[Old broken answer has been edited, so the following may not match the comments below]

In props.conf, you can specify the source using a regular expression (as well as with the "normal" wildcards). Therefore, this should work.

[source::/opt/splunk/var/log/syslog/.*?lx0001.*?/.*]

The above stanza should match exactly what you want. But it would be even better if you replaced the .*? with more precise regex matching; this is pretty open-ended.

When you have time (it is long), read the header of the props.conf.spec file carefully and you will find a wealth of good information.

Re: How can I use wildcards (*) for the source stanza in props.conf?

jonatanjosefson — Wed, 28 Oct 2015 07:51:19 GMT

Unfortunately, neither work.

I also cannot find any official info that you're able to use whitelist in props.conf

Re: How can I use wildcards (*) for the source stanza in props.conf?

lguinn2 — Wed, 28 Oct 2015 23:48:18 GMT

Oh - shoot me now! My head was in inputs.conf while I was clearly working with props.conf

Above answer is completely revised. Thanks!

Re: How can I use wildcards (*) for the source stanza in props.conf?

jonatanjosefson — Thu, 29 Oct 2015 14:35:58 GMT

Yeah that regex checks out on my different sources testing it outside Splunk but it doesn't work in my props.conf. Are you really able to write regex simply like that in the source stanza?

Thanks for your time.

Re: How can I use wildcards (*) for the source stanza in props.conf?

landen99 — Tue, 16 Jul 2019 13:30:01 GMT

https://answers.splunk.com/answers/397051/can-you-regex-or-wildcard-props-sourcetype-stanzas.html

Re: How can I use wildcards (*) for the source stanza in props.conf?

splunkreal — Tue, 19 Nov 2024 11:30:54 GMT

Hello @somesoni2 @lguinn2 @landen99 @jonatanjosefson any solution on this? I want to specify source::mylog_*

Thanks for your help!

Re: How can I use wildcards (*) for the source stanza in props.conf?

lguinn2 — Tue, 19 Nov 2024 21:19:05 GMT

For the source stanza, Splunk uses regular expressions that are PCRE (Perl Compatible Regular Expressions).

From props.conf.spec

**[source::<source>] and [host::<host>] stanza match language:**

Match expressions must match the entire name, not just a substring. Match
expressions are based on a full implementation of Perl-compatible regular
expressions (PCRE) with the translation of "...", "*", and "." Thus, "."
matches a period, "*" matches non-directory separators, and "..." matches
any number of any characters.

Also from props.conf.spec

When setting a [<spec>] stanza, you can use the following regex-type syntax:
... recurses through directories until the match is met
    or equivalently, matches any number of characters.
*   matches anything but the path separator 0 or more times.
    The path separator is '/' on unix, or '\' on Windows.
    Intended to match a partial or complete directory or filename.
|   is equivalent to 'or'
( ) are used to limit scope of |.
\\ = matches a literal backslash '\'.

So for mylog_*

you could specify

source::.../mylog_*

It's been a few years on this one, so hope I am right this time!

Re: How can I use wildcards (*) for the source stanza in props.conf?

lguinn2 — Tue, 19 Nov 2024 21:33:54 GMT

Let's try one last time!

In props.conf, the source stanza actually follows a modified regex syntax. (See my other comment from 19-nov-2024 or just read props.conf.spec). So the following should work

[source::/opt/splunk/var/log/syslog/*lx0001*/*]
or perhaps
[source::/opt/splunk/var/log/syslog/*lx0001*/*.log]

But re-reading the original post, @jonatanjosefson was actually trying to set some property for all syslog events corresponding to a particular host name pattern - you can do that using a host stanza instead of a source stanza:

host::*lx0001*

To me, this seems easier to understand and maintain, as it will work even if the directory structure of the syslog files changes over time. It depends only on the hostname, and not the file name or location.