I have the Splunk Add-on for NetFlow installed on Splunk Heavy Forwarder (HF), receiving data from Netflow enabled device. I am getting the data and it is sent successfully to indexer. I want to filter fields send only specific fields for indexing. Can you let me know how to achieve this?
E.g I need to completely remove below fields from indexing. These fields are as is extracted by add-on
Will the below configuration work? And do I have to perform the change at HF or Indexer level?
[netflow] TRANSFORMS-nullQ= nullFilter
[nullFilter] REGEX = tcp_flag|fwd_status|src_tos DEST_KEY=queue FORMAT = nullQueue
Do you need to:
1) Stop specific fields from being created but allow the events that have that data to be forwarded in?
2) Stop the events themselves from coming into Splunk?
Thanks for your response.
Fields were created by Netflow add on (installed on HF) based on the data and it has pre-defined extraction. Data comes in binary format which Add-on converts into ascii format and send to indexers.
I want to stop the events for some specific fields to send to Indexer. For e.g. I don't want tcpflag,fwdstatus,src_tos field data to be sent to indexer. There are other fields too which i want to filter out. Based on this sample , I can work them out.
Appreciate your help.
I should have added this:
3) Strip the raw events of certain bits of data but let the rest of the event string come in.
I think you mean #3 but I need you to confirm. I do not at all understand the way that you have phrased the question.
Yes that is correct, I want to strip certain raw events (fields) from the event and then send to indexer.
Unfortunately , I will not be able to use Splunk Stream for now.
@hemendralodhi Now showing in a theater near you:
Splunk Stream 7: The Filtered Dataset. Download now at Splunkbase [https://splunkbase.splunk.com/app/1809]. Splunk Stream captures Netflow (v5, v9, and IPFIX) and allows you to filter based on the fields being sent in the data AS WELL AS allow you to pick which fields to index. Without using a props or transforms! Read the full documentation here: http://docs.splunk.com/Documentation/StreamApp/7.0.1/DeployStreamApp/ConfigureFlowcollector
This is to be used in the place of the Netflow Add-On, as it is more robust and scalable.
Thanks for the recommendation , but I may not be able to use Stream for now but look into it later.
Stream is really your only option here. You cannot simply and easily remove specific fields from the Netflow traffic using
transforms. You can drop ENTIRE events, yes, but you cannot pull out specific fields. You need stream to do that, otherwise you will be writing some very Nasty