filtering data before indexer

[abrice]

I'm trying to filter data before it reaches the indexers ON the index server (I'm using universal forwarders...). I've tested on regex101 and verified the regex is good. I'm still seeing data coming in from the 4 IP's.

I have this setup:

props.conf
[Filter_Logs]
TRANSFORMS-null = null,Filter_Logs

transforms.conf
[null]
REGEX = .
DEST_KEY = queue
FORMAT = indexQueue

[Filter_Logs]
REGEX = (10.1.1.1 -|10.1.1.2 -|10.2.1.1 - |10.2.1.2 -)
DEST_KEY = queue
FORMAT = nullQueue

[abrice]

Thank you for the reply.

I'm trying to put the filtering on the indexers and not the forwarders as we already have the forwarders deployed and don't want to change that. From what I understand the filtering can be done in either location, either at the forwarders or on the index servers.

[sudosplunk]

IMO, these configurations should be pushed to both UF and Indexer(s). I've seen filtering not working properly if configs are not present on both source and destination. Worth trying!

[abrice]

Just tried that on one web server, no luck.

on the indexers I've tried it via pushing from the cluster master as an app as well as putting it in etc/system/local

[sudosplunk]

Hello, were you able to solve this? If you sense everything is correct, then please revisit your regex again. In your regex (the one in question), I noticed there are whitespace characters which seems to be little off. Is it possible to provide some sample events?

[pruthvikrishnap]

Hi Abrice,

UF do not have the capability which you are looking for, you need to replace your current universal forwarder with a heavy forwarder. Can copy all of your configuration files from the UF to the HF without having to rewrite anything.
Below article helps you configure.

http://docs.splunk.com/Documentation/Splunk/6.2.2/Forwarding/Routeandfilterdatad
Let me know if this helps.