Splunk Search

How to fix my regex configuration to filter out events properly before indexing?


Hello Everyone,

After doing quite a bit of research I believe I have the correct process for filtering out information before it is indexed however the traffic is still being indexed. 1 source is sending 3 types of information. One of those source types is network_traffic, which we want to not be indexed.

The sourcetype = network_traffic

cat /opt/splunk/etc/system/local/props.conf 

cat /opt/splunk/etc/system/local/transforms.conf
REGEX = .*
DEST_KEY = queue
FORMAT = nullQueue

Any thoughts or ideas would be great. I've tried using specific regex to match on traffic inside the index, putting spaces / removing spaces before and after = signs, etc.



As a FYI. This traffic is being forwarded already cooked from another indexer. We didn't think this would be an issue however considering it is already cooked it could be?

0 Karma


If the data has already gone through the TRANSFORMS pipeline on the HF, (hence your comment about it already being "cooked"), then you are correct, you only get ONE pass through that data pipeline.

If you process the data using props.conf and transforms.conf on the HF, you do not get a second pass to do it again at the indexer tier.

0 Karma

I believe that you will need to do this filtering at your heavy forwarder or wherever the data is getting cooked.

0 Karma


Excuse me if i did not understand correctly. The events you to null, are they multiline?
(?s).* maybe

0 Karma


I do not believe they are multiline, and believe they are simply one long line. I did attempt to use (?s).* and din't have any luck.

0 Karma


Have you tried REGEX = (.*)?

If this reply helps you, an upvote would be appreciated.
0 Karma


I had not, however I just did and still seems to have the same issue.

0 Karma
Don’t Miss Global Splunk
User Groups Week!

Free LIVE events worldwide 2/8-2/12
Connect, learn, and collect rad prizes
and swag!