Hi, how to exclude internal source IP events for a sourcetype (web_logs) with src_ip=10.0.0.0/8 before indexing.
You can actually save some licensing too, just by blacklisting the field, along with it's value in inputs.conf itself.
blacklist = src_ip = 10.0.0.0/8
Else, routing the data to null queue, as explained by @mguhad, will work too. I find blacklisting the data to be more helpful, when i know I don't need a portion of data at all. Why even bother to ingest it, when it has no use.
You can eliminate unwanted data by routing it to the nullQueue and it is removed before indexing with a combination of props and tranforms .conf like below:
props.conf:
[web_logs]
TRANSFORMS-removeIP= exclude
transforms.conf:
[exclude]
REGEX = src_ip=\d{2}\.\d{1}\.\d{1}\.\d{1}\/8$
DEST_KEY = queue
FORMAT = nullQueue
For methods to keep segments of the data and discard the rest, please the docs here:https://docs.splunk.com/Documentation/Splunk/8.0.1/Forwarding/Routeandfilterdatad