How do forwardedindex whitelists and blacklists work?

[hettervik]

Hi,

I'm trying to figure out how the whitelist and blacklist in outputs.conf work. By default it looks like this:

 [tcpout]
 forwardedindex.0.whitelist = .*
 forwardedindex.1.blacklist = _.*
 forwardedindex.2.whitelist = (_audit|_internal)

It is stated in the doc that blacklist overrides whitelist. Doesn't this mean that rule #1 overrides rule #2? Doesn't rule #1 state that all files starting with an underscore is blacklisted? Still rule #2 states that _audit and _internal are whitelisted. As far as I know, internal and audit are forwarded by default. Can someone help me understand this?

[renjith_nair · Accepted solution]

Have a look at the outputs.conf

#----Index Filter Settings.
# These attributes are only applicable under the global [tcpout] stanza. This filter does not work if it is created 
# under any other stanza.
forwardedindex.<n>.whitelist = <regex>
forwardedindex.<n>.blacklist = <regex>
* These filters determine which events get forwarded, based on the indexes they belong to.
* This is an ordered list of whitelists and blacklists, which together decide if events should be forwarded to an index.
* The order is determined by <n>. <n> must start at 0 and continue with positive integers, in sequence. There cannot be any
  gaps in the sequence. (For example, forwardedindex.0.whitelist, forwardedindex.1.blacklist, forwardedindex.2.whitelist, ...). 
* The filters can start from either whitelist or blacklist. They are tested from forwardedindex.0 to forwardedindex.<max>.
*** If both forwardedindex.<n>.whitelist and forwardedindex.<n>.blacklist are present for the same value of n, then
  forwardedindex.<n>.whitelist is honored. forwardedindex.<n>.blacklist is ignored in this case.**
* You should not normally need to change these filters from their default settings in $SPLUNK_HOME/system/default/outputs.conf.
* Filtered out events are not indexed if local indexing is not enabled.

The numbers n determines which one should be considered first. For eg: in your above example when splunk parses it

0.Accepts all
1.Exclude _ indexes
2.Accept audit and internal

If you alter the order 1 and 2 , ie

forwardedindex.0.whitelist = .*
forwardedindex.1.whitelist = (_audit|_internal)
forwardedindex.2.blacklist = _.*

0 Accept all
1 Accept audit and internal
2 Reject all internal

In this case it will not forward as rule#2 overrides rule#1

[hettervik]

Thanks a lot, that made things clearer! I've used Cisco systems before, and I'm used to access lists where you read from the top down and use the first rule that applies to your current file. Of course this wouldn't make sense here, where the first default rule is an "accept all"-rule.