How to fix my regex configuration to filter out events properly before indexing?

[jstaley]

Hello Everyone,

After doing quite a bit of research I believe I have the correct process for filtering out information before it is indexed however the traffic is still being indexed. 1 source is sending 3 types of information. One of those source types is network_traffic, which we want to not be indexed.

The sourcetype = network_traffic

cat /opt/splunk/etc/system/local/props.conf 
[network_traffic]
TRANSFORMS-null=traffic_null

cat /opt/splunk/etc/system/local/transforms.conf
[traffic_null]
REGEX = .*
DEST_KEY = queue
FORMAT = nullQueue

Any thoughts or ideas would be great. I've tried using specific regex to match on traffic inside the index, putting spaces / removing spaces before and after = signs, etc.

Thanks!

[Edit]

As a FYI. This traffic is being forwarded already cooked from another indexer. We didn't think this would be an issue however considering it is already cooked it could be?

[joelyon]

If the data has already gone through the TRANSFORMS pipeline on the HF, (hence your comment about it already being "cooked"), then you are correct, you only get ONE pass through that data pipeline.

If you process the data using props.conf and transforms.conf on the HF, you do not get a second pass to do it again at the indexer tier.

[richard_g_curry]

I believe that you will need to do this filtering at your heavy forwarder or wherever the data is getting cooked.

[Raghav2384]

Excuse me if i did not understand correctly. The events you to null, are they multiline?
(?s).* maybe
thanks,
raghav

[jstaley]

I do not believe they are multiline, and believe they are simply one long line. I did attempt to use (?s).* and din't have any luck.

[richgalloway]

Have you tried REGEX = (.*)?

[jstaley]

I had not, however I just did and still seems to have the same issue.