Setting a lower priority in props.conf

[Lowell]

How do you properly set a source matching stanza in props to be lower than the default stanza matching priority?

Per the docs, priority=0 is given to all pattern patching, and a priority of 100 is given to all explicit file names.

I'm having an issue with sourcetypes being assigned to files under /etc/. The default rule in the unix app left splunk to make up a bunch of it's own new sourcetypes (which can be quite annoying) so I added a [source:/etc/....] rule to assign the sourcetype of config_file, but I'm hitting a few issues because of that.

Here are the three types of source-matching rules I'm trying to manage:

1.) I mostly want files under /etc/... to be indexed with the sourcetype config_file. I really want this to be just the default fall-back value if no other pattern matches.

2.) There are a few sensitive files that should never be index, such as /etc/passwd* or /etc/shadow*. So these should be assigned the ignored_type sourcetype. (Since this is a wildcard expression it gets set with priority=0, so we have to bump the priority, and it seems wise to set it to 100 or higher.)

3.) There are some more specific patterns, for example *.xml where I would rather use the sourcetype of xml_file rather than config_file. However both /etc/* and *.xml are given the same priority=0. So I could bump *.xml to a higher priority, but then I end up having to go around and also bump the priority of a bunch of other more-specific rules. (For example, I don't want for example want a *.xml rule to win out over a .../my_application/config.xml rule where I want the sourcetype to be my_app_config, not just xml_file.)

The most obvious answer to me was to simply assign the [source::/etc/...] rule a lower priority. However, since the priority is already 0 the next lowest priority is -1, and while I thought that was working; it now seems to win out over my priority=100 entries; hence it's possible to index my password files, which is highly undesirable. (I'm guessing this isn't supported. This may have happened when I upgraded to 4.1.3, but I don't remember for sure checking it in 4.1.2)

So I'm trying to figure out the best way to handle this. If I have to go around and re-prioritize all my source matching stanzas, then I'm okay with that, event though it will most likely start a chain reaction that will require me to update all of my 15+ custom apps at once to restore the proper balance of power in my own little splunk world. I do also have a concern about how to handle this if I run into a situation where I want to bump one of splunk's built-in source matching rules to a higher priority. I've run into problems like this before when using the "local" props files to override splunk's "default" settings and the stanza names changes. (For example, in 4.1.x splunk renamed the WMI script input from a .py file to a .path file so my custom index=os setting didn't' take effect because the stanza name changed. In 4.1 all of the input scripts in the unix changed stanza names too.) I understand that this kind of stuff is necessary at times, but this stuff takes some time to identify and get fixed after each upgrade. And each minor tweak is just adding to the total pain of each successive upgrade, so I don't want to make it worse by overriding priorities of built-in source matching stanzas.

---

To summarize...

• Is it possible to assign a priority lower than 0? (or bump all the undeclared priorities to a higher value?)
• Are there any best-practice or recommendations on assigning priorities to source patterns?
• Anything I'm missing here? Am I making this too complicated?

[jrodman]

I initially advocated having a priority=high, prioriority=low and everything at priorty=default without anything explicit being between the two. For precisely this reason.

That's not what we built. We didn't provide a config-loser value.

If -1 wins over 100 it's probably being stored as MAXINT, which is an out and out bug. Sorry you encountered it. Thanks for reporting it.

[jrodman]

Possibly you could set priority=1 in system/local/props.conf in [default]. There's some reasons you might not want priority to flow through though, so it might be a special case, but wouldn't expect so.

[Lowell]

I was wondering if that was happening. (Or event if it was an unsigned int kind of thing.) So you don't think it's possible to bump up the default "0" priority? using a config setting?