All Apps and Add-ons
Highlighted

Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer?

Contributor

Hello Team,

I have the Splunk Add-on for NetFlow installed on Splunk Heavy Forwarder (HF), receiving data from Netflow enabled device. I am getting the data and it is sent successfully to indexer. I want to filter fields send only specific fields for indexing. Can you let me know how to achieve this?

E.g I need to completely remove below fields from indexing. These fields are as is extracted by add-on

tcp_flag,fwd_status,src_tos

Will the below configuration work? And do I have to perform the change at HF or Indexer level?

props.conf

[netflow]
TRANSFORMS-nullQ= nullFilter

transforms.conf

[nullFilter]
REGEX = tcp_flag|fwd_status|src_tos
DEST_KEY=queue
FORMAT = nullQueue

Thanks
Hemendra

0 Karma
Highlighted

Re: Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer?

Esteemed Legend

Do you need to:
1) Stop specific fields from being created but allow the events that have that data to be forwarded in?
2) Stop the events themselves from coming into Splunk?

0 Karma
Highlighted

Re: Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer?

Contributor

Hello Woodcock,

Thanks for your response.

Fields were created by Netflow add on (installed on HF) based on the data and it has pre-defined extraction. Data comes in binary format which Add-on converts into ascii format and send to indexers.

I want to stop the events for some specific fields to send to Indexer. For e.g. I don't want tcpflag,fwdstatus,src_tos field data to be sent to indexer. There are other fields too which i want to filter out. Based on this sample , I can work them out.

Appreciate your help.

Thanks
Hemendra

0 Karma
Highlighted

Re: Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer?

Esteemed Legend

I should have added this:
3) Strip the raw events of certain bits of data but let the rest of the event string come in.

I think you mean #3 but I need you to confirm. I do not at all understand the way that you have phrased the question.

0 Karma
Highlighted

Re: Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer?

Contributor

Hello Woodcock,

Yes that is correct, I want to strip certain raw events (fields) from the event and then send to indexer.

Please advise.

Unfortunately , I will not be able to use Splunk Stream for now.

0 Karma
Highlighted

Re: Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer?

Esteemed Legend

See the answer by @alacercogitatus. He is spot on.

0 Karma
Highlighted

Re: Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer?

SplunkTrust
SplunkTrust

@hemendralodhi Now showing in a theater near you: Splunk Stream 7: The Filtered Dataset. Download now at Splunkbase [https://splunkbase.splunk.com/app/1809]. Splunk Stream captures Netflow (v5, v9, and IPFIX) and allows you to filter based on the fields being sent in the data AS WELL AS allow you to pick which fields to index. Without using a props or transforms! Read the full documentation here: http://docs.splunk.com/Documentation/StreamApp/7.0.1/DeployStreamApp/ConfigureFlowcollector

This is to be used in the place of the Netflow Add-On, as it is more robust and scalable.

Highlighted

Re: Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer?

Contributor

Thanks for the recommendation , but I may not be able to use Stream for now but look into it later.

0 Karma
Highlighted

Re: Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer?

SplunkTrust
SplunkTrust

Stream is really your only option here. You cannot simply and easily remove specific fields from the Netflow traffic using props and transforms. You can drop ENTIRE events, yes, but you cannot pull out specific fields. You need stream to do that, otherwise you will be writing some very Nasty SEDCMD settings.

0 Karma
Highlighted

Re: Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer?

Esteemed Legend
0 Karma