After doing quite a bit of research I believe I have the correct process for filtering out information before it is indexed however the traffic is still being indexed. 1 source is sending 3 types of information. One of those source types is network_traffic, which we want to not be indexed.
The sourcetype = network_traffic
cat /opt/splunk/etc/system/local/props.conf [network_traffic] TRANSFORMS-null=traffic_null cat /opt/splunk/etc/system/local/transforms.conf [traffic_null] REGEX = .* DEST_KEY = queue FORMAT = nullQueue
Any thoughts or ideas would be great. I've tried using specific regex to match on traffic inside the index, putting spaces / removing spaces before and after = signs, etc.
As a FYI. This traffic is being forwarded already cooked from another indexer. We didn't think this would be an issue however considering it is already cooked it could be?
If the data has already gone through the TRANSFORMS pipeline on the HF, (hence your comment about it already being "cooked"), then you are correct, you only get ONE pass through that data pipeline.
If you process the data using props.conf and transforms.conf on the HF, you do not get a second pass to do it again at the indexer tier.