How do you properly set a source matching stanza in props to be lower than the default stanza matching priority?
Per the docs, priority=0 is given to all pattern patching, and a priority of 100 is given to all explicit file names.
I'm having an issue with sourcetypes being assigned to files under
/etc/. The default rule in the
unix app left splunk to make up a bunch of it's own new sourcetypes (which can be quite annoying) so I added a
[source:/etc/....] rule to assign the sourcetype of
config_file, but I'm hitting a few issues because of that.
1.) I mostly want files under
/etc/... to be indexed with the sourcetype
config_file. I really want this to be just the default fall-back value if no other pattern matches.
2.) There are a few sensitive files that should never be index, such as
/etc/shadow*. So these should be assigned the
ignored_type sourcetype. (Since this is a wildcard expression it gets set with
priority=0, so we have to bump the priority, and it seems wise to set it to 100 or higher.)
3.) There are some more specific patterns, for example
*.xml where I would rather use the sourcetype of
xml_file rather than
config_file. However both
*.xml are given the same priority=0. So I could bump
*.xml to a higher priority, but then I end up having to go around and also bump the priority of a bunch of other more-specific rules. (For example, I don't want for example want a
*.xml rule to win out over a
.../my_application/config.xml rule where I want the sourcetype to be
my_app_config, not just
The most obvious answer to me was to simply assign the
[source::/etc/...] rule a lower priority. However, since the priority is already
0 the next lowest priority is
-1, and while I thought that was working; it now seems to win out over my priority=100 entries; hence it's possible to index my password files, which is highly undesirable. (I'm guessing this isn't supported. This may have happened when I upgraded to 4.1.3, but I don't remember for sure checking it in 4.1.2)
So I'm trying to figure out the best way to handle this. If I have to go around and re-prioritize all my source matching stanzas, then I'm okay with that, event though it will most likely start a chain reaction that will require me to update all of my 15+ custom apps at once to restore the proper balance of power in my own little splunk world. I do also have a concern about how to handle this if I run into a situation where I want to bump one of splunk's built-in source matching rules to a higher priority. I've run into problems like this before when using the "local" props files to override splunk's "default" settings and the stanza names changes. (For example, in 4.1.x splunk renamed the WMI script input from a
.py file to a
.path file so my custom
index=os setting didn't' take effect because the stanza name changed. In 4.1 all of the input scripts in the
unix changed stanza names too.) I understand that this kind of stuff is necessary at times, but this stuff takes some time to identify and get fixed after each upgrade. And each minor tweak is just adding to the total pain of each successive upgrade, so I don't want to make it worse by overriding priorities of built-in source matching stanzas.
I initially advocated having a priority=high, prioriority=low and everything at priorty=default without anything explicit being between the two. For precisely this reason.
That's not what we built. We didn't provide a config-loser value.
If -1 wins over 100 it's probably being stored as MAXINT, which is an out and out bug. Sorry you encountered it. Thanks for reporting it.
I was wondering if that was happening. (Or event if it was an unsigned int kind of thing.) So you don't think it's possible to bump up the default "0" priority? using a config setting?
Possibly you could set priority=1 in system/local/props.conf in [default]. There's some reasons you might not want priority to flow through though, so it might be a special case, but wouldn't expect so.