topic Re: Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer? in All Apps and Add-ons

Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer?

hemendralodhi — Mon, 20 Mar 2017 05:42:44 GMT

Hello Team,

I have the Splunk Add-on for NetFlow installed on Splunk Heavy Forwarder (HF), receiving data from Netflow enabled device. I am getting the data and it is sent successfully to indexer. I want to filter fields send only specific fields for indexing. Can you let me know how to achieve this?

E.g I need to completely remove below fields from indexing. These fields are as is extracted by add-on

tcp_flag,fwd_status,src_tos

Will the below configuration work? And do I have to perform the change at HF or Indexer level?

props.conf

[netflow]
TRANSFORMS-nullQ= nullFilter

transforms.conf

[nullFilter]
REGEX = tcp_flag|fwd_status|src_tos
DEST_KEY=queue
FORMAT = nullQueue

Thanks
Hemendra

Re: Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer?

woodcock — Mon, 20 Mar 2017 16:54:45 GMT

Do you need to:
1) Stop specific fields from being created but allow the events that have that data to be forwarded in?
2) Stop the events themselves from coming into Splunk?

Re: Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer?

hemendralodhi — Tue, 29 Sep 2020 13:16:42 GMT

Hello Woodcock,

Thanks for your response.

Fields were created by Netflow add on (installed on HF) based on the data and it has pre-defined extraction. Data comes in binary format which Add-on converts into ascii format and send to indexers.

I want to stop the events for some specific fields to send to Indexer. For e.g. I don't want tcp_flag,fwd_status,src_tos field data to be sent to indexer. There are other fields too which i want to filter out. Based on this sample , I can work them out.

Appreciate your help.

Thanks
Hemendra

Re: Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer?

woodcock — Tue, 21 Mar 2017 01:16:25 GMT

I should have added this:
3) Strip the raw events of certain bits of data but let the rest of the event string come in.

I think you mean #3 but I need you to confirm. I do not at all understand the way that you have phrased the question.

Re: Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer?

alacercogitatus — Tue, 21 Mar 2017 13:36:55 GMT

@hemendralodhi Now showing in a theater near you: Splunk Stream 7: The Filtered Dataset. Download now at Splunkbase [https://splunkbase.splunk.com/app/1809]. Splunk Stream captures Netflow (v5, v9, and IPFIX) and allows you to filter based on the fields being sent in the data AS WELL AS allow you to pick which fields to index. Without using a props or transforms! Read the full documentation here: http://docs.splunk.com/Documentation/StreamApp/7.0.1/DeployStreamApp/ConfigureFlowcollector

This is to be used in the place of the Netflow Add-On, as it is more robust and scalable.

Re: Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer?

woodcock — Tue, 21 Mar 2017 14:20:43 GMT

See the answer by @alacercogitatus. He is spot on.

Re: Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer?

hemendralodhi — Tue, 21 Mar 2017 19:58:28 GMT

Hello Woodcock,

Yes that is correct, I want to strip certain raw events (fields) from the event and then send to indexer.

Please advise.

Unfortunately , I will not be able to use Splunk Stream for now.

Re: Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer?

hemendralodhi — Tue, 21 Mar 2017 19:59:26 GMT

Thanks for the recommendation , but I may not be able to use Stream for now but look into it later.

Re: Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer?

woodcock — Tue, 21 Mar 2017 20:01:02 GMT

Actually (without the period):

http://docs.splunk.com/Documentation/StreamApp/7.0.1/DeployStreamApp/ConfigureFlowcollector

Re: Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer?

alacercogitatus — Tue, 21 Mar 2017 20:01:18 GMT

Stream is really your only option here. You cannot simply and easily remove specific fields from the Netflow traffic using props and transforms. You can drop ENTIRE events, yes, but you cannot pull out specific fields. You need stream to do that, otherwise you will be writing some very Nasty SEDCMD settings.

Re: Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer?

alacercogitatus — Tue, 21 Mar 2017 20:02:06 GMT

edited thanks!