Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer?

[hemendralodhi]

Hello Team,

I have the Splunk Add-on for NetFlow installed on Splunk Heavy Forwarder (HF), receiving data from Netflow enabled device. I am getting the data and it is sent successfully to indexer. I want to filter fields send only specific fields for indexing. Can you let me know how to achieve this?

E.g I need to completely remove below fields from indexing. These fields are as is extracted by add-on

tcp_flag,fwd_status,src_tos

Will the below configuration work? And do I have to perform the change at HF or Indexer level?

props.conf

[netflow]
TRANSFORMS-nullQ= nullFilter

transforms.conf

[nullFilter]
REGEX = tcp_flag|fwd_status|src_tos
DEST_KEY=queue
FORMAT = nullQueue

Thanks
Hemendra

[alacercogitatus]

@hemendralodhi Now showing in a theater near you: Splunk Stream 7: The Filtered Dataset. Download now at Splunkbase [https://splunkbase.splunk.com/app/1809]. Splunk Stream captures Netflow (v5, v9, and IPFIX) and allows you to filter based on the fields being sent in the data AS WELL AS allow you to pick which fields to index. Without using a props or transforms! Read the full documentation here: http://docs.splunk.com/Documentation/StreamApp/7.0.1/DeployStreamApp/ConfigureFlowcollector

This is to be used in the place of the Netflow Add-On, as it is more robust and scalable.

[woodcock]

Actually (without the period):

http://docs.splunk.com/Documentation/StreamApp/7.0.1/DeployStreamApp/ConfigureFlowcollector

[alacercogitatus]

edited thanks!

[hemendralodhi]

Thanks for the recommendation , but I may not be able to use Stream for now but look into it later.

[alacercogitatus]

Stream is really your only option here. You cannot simply and easily remove specific fields from the Netflow traffic using props and transforms. You can drop ENTIRE events, yes, but you cannot pull out specific fields. You need stream to do that, otherwise you will be writing some very Nasty SEDCMD settings.

[woodcock]

Do you need to:
1) Stop specific fields from being created but allow the events that have that data to be forwarded in?
2) Stop the events themselves from coming into Splunk?

[hemendralodhi]

Hello Woodcock,

Thanks for your response.

Fields were created by Netflow add on (installed on HF) based on the data and it has pre-defined extraction. Data comes in binary format which Add-on converts into ascii format and send to indexers.

I want to stop the events for some specific fields to send to Indexer. For e.g. I don't want tcp_flag,fwd_status,src_tos field data to be sent to indexer. There are other fields too which i want to filter out. Based on this sample , I can work them out.

Appreciate your help.

Thanks
Hemendra

[woodcock]

See the answer by @alacercogitatus. He is spot on.

[woodcock]

I should have added this:
3) Strip the raw events of certain bits of data but let the rest of the event string come in.

I think you mean #3 but I need you to confirm. I do not at all understand the way that you have phrased the question.

[hemendralodhi]

Hello Woodcock,

Yes that is correct, I want to strip certain raw events (fields) from the event and then send to indexer.

Please advise.

Unfortunately , I will not be able to use Splunk Stream for now.