Getting Data In

Difficulty Understanzing Stanza in props.conf

TheColorBlack
Explorer

Hey all,

I have a relatively dumb question. I'm trying to familiarize myself with Splunk's props.conf and transforms.conf files. Within one of my props.conf file there's a stanza defined for a custom sourcetype of mine but I can't figure out how the stanza is formatted as dumb as that sounds.

 

The stanza in props.conf is defined as

[(::){0}json:myCustomSourceType:*]

 

Could anyone help me understand what the "(::){0}" portion of that stanza is defining? According to the documentation for props.conf the accepted stanza formats are

 

[<spec>]
* This stanza enables properties for a given <spec>.
* A props.conf file can contain multiple stanzas for any number of
  different <spec>.
* Follow this stanza name with any number of the following setting/value
  pairs, as appropriate for what you want to do.
* If you do not set a setting for a given <spec>, the default is used.

<spec> can be:
1. <sourcetype>, the source type of an event.
2. host::<host>, where <host> is the host, or host-matching pattern, for an
                 event.
3. source::<source>, where <source> is the source, or source-matching
                     pattern, for an event.
4. rule::<rulename>, where <rulename> is a unique name of a source type
                     classification rule.
5. delayedrule::<rulename>, where <rulename> is a unique name of a delayed
                            source type classification rule.
                            These are only considered as a last resort
                            before generating a new source type based on the
                            source seen.

 

Labels (2)
0 Karma
1 Solution

TheColorBlack
Explorer

Sorry to bump this thread up from the grave but I recently figured this out.

 

 

# Example props.conf file

[(::){0}myCustomSourceType*]

 

 

The "(::){0}" portion of the stanza is an unsupported hack to force Splunk to support regular expressions / wild-card matching for SourceType Stanzas within props.conf.

I'm going out on a limb here for my explication. I believe "(::){0}" defines a regex that equals "match :: zero or more times" which would in turn directly correlate to the long hand sourcetype, rule, host, delayed-rule definition outlined in the props.conf documentation.

 

<spec> can be:
1. <sourcetype>, the source type of an event.
2. host::<host>, where <host> is the host, or host-matching pattern, for an
                 event.
3. source::<source>, where <source> is the source, or source-matching
                     pattern, for an event.
4. rule::<rulename>, where <rulename> is a unique name of a source type
                     classification rule.
5. delayedrule::<rulename>, where <rulename> is a unique name of a delayed
                            source type classification rule.
                            These are only considered as a last resort
                            before generating a new source type based on the
                            source seen.

 

 

Please keep in mind that this is an unsupported method according to Splunk.

From The Splunk Community Thread "Can you have a wildcard in a props.conf stanza header when matching sourcetypes?"

This same solution is presented in a 2014 Splunk blog article written by Splunk's Jason Conger. However, Splunk's Joshua Rodman commented: "This is not supported product functionality, and relying on it is extremely unwise."

-------------------------------------------------------------------------

From The Splunk Community Thread "Can you regex or wildcard props Sourcetype stanzas?"

Sourcetypes stanza in props.conf do not support regexes matching.
The answer above is an hack, that is not supported and non maintained by splunk codebase.

So do not use and if it works by sheer luck, do not expect it to work in all versions and all situations. ( e.g. do not rely on that hack for any critical business usage)

-------------------------------------------------------------------------

From The Splunk Community Thread "Is it possible to use wildcards in sourcetype props.conf stanzas"

I don't think we should be recommending this hack very much. If this is a common need, then make Splunk add proper support for wildcarding sourcetypes. This above thing relies on a very internal interaction between the regex expression and the way the matching is accomplished and it could easily change in the future.

It does work currently, but I would definitely recommend against leaning on it heavily -- leaving it installed for other parties at customer sites, putting it in apps, leaving an employer as a splunk admin where critical functionality relies on this without specifically passing on that information, etc, are all things that may do others a disservice in the longer run.

However, as of Splunk version 8.0.2 this method does work. As tedious as it is, the supported method is to define an explicit stanza for each of your custom sourcetypes within your props.conf file, even if the sourcetypes all have a similar naming scheme / purpose.

 

References:

1) Can You Have a Wildcard in a Props.conf Stanza Header When Matching Sourcetypes? 
2) Trick to Using Wildcards in Props.conf 
3) QuickTip: Using Wildcard Sourcetypes in Props.Conf 
4) Is It Possible to use wildcards in sourcetype props.conf stanzas? 
5) Can you Regex or Wildcard Props.conf Sourcetype Stanzas? 

View solution in original post

0 Karma

TheColorBlack
Explorer

Sorry to bump this thread up from the grave but I recently figured this out.

 

 

# Example props.conf file

[(::){0}myCustomSourceType*]

 

 

The "(::){0}" portion of the stanza is an unsupported hack to force Splunk to support regular expressions / wild-card matching for SourceType Stanzas within props.conf.

I'm going out on a limb here for my explication. I believe "(::){0}" defines a regex that equals "match :: zero or more times" which would in turn directly correlate to the long hand sourcetype, rule, host, delayed-rule definition outlined in the props.conf documentation.

 

<spec> can be:
1. <sourcetype>, the source type of an event.
2. host::<host>, where <host> is the host, or host-matching pattern, for an
                 event.
3. source::<source>, where <source> is the source, or source-matching
                     pattern, for an event.
4. rule::<rulename>, where <rulename> is a unique name of a source type
                     classification rule.
5. delayedrule::<rulename>, where <rulename> is a unique name of a delayed
                            source type classification rule.
                            These are only considered as a last resort
                            before generating a new source type based on the
                            source seen.

 

 

Please keep in mind that this is an unsupported method according to Splunk.

From The Splunk Community Thread "Can you have a wildcard in a props.conf stanza header when matching sourcetypes?"

This same solution is presented in a 2014 Splunk blog article written by Splunk's Jason Conger. However, Splunk's Joshua Rodman commented: "This is not supported product functionality, and relying on it is extremely unwise."

-------------------------------------------------------------------------

From The Splunk Community Thread "Can you regex or wildcard props Sourcetype stanzas?"

Sourcetypes stanza in props.conf do not support regexes matching.
The answer above is an hack, that is not supported and non maintained by splunk codebase.

So do not use and if it works by sheer luck, do not expect it to work in all versions and all situations. ( e.g. do not rely on that hack for any critical business usage)

-------------------------------------------------------------------------

From The Splunk Community Thread "Is it possible to use wildcards in sourcetype props.conf stanzas"

I don't think we should be recommending this hack very much. If this is a common need, then make Splunk add proper support for wildcarding sourcetypes. This above thing relies on a very internal interaction between the regex expression and the way the matching is accomplished and it could easily change in the future.

It does work currently, but I would definitely recommend against leaning on it heavily -- leaving it installed for other parties at customer sites, putting it in apps, leaving an employer as a splunk admin where critical functionality relies on this without specifically passing on that information, etc, are all things that may do others a disservice in the longer run.

However, as of Splunk version 8.0.2 this method does work. As tedious as it is, the supported method is to define an explicit stanza for each of your custom sourcetypes within your props.conf file, even if the sourcetypes all have a similar naming scheme / purpose.

 

References:

1) Can You Have a Wildcard in a Props.conf Stanza Header When Matching Sourcetypes? 
2) Trick to Using Wildcards in Props.conf 
3) QuickTip: Using Wildcard Sourcetypes in Props.Conf 
4) Is It Possible to use wildcards in sourcetype props.conf stanzas? 
5) Can you Regex or Wildcard Props.conf Sourcetype Stanzas? 

View solution in original post

0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!