topic Re: Stop indexing particular logs in Getting Data In

Stop indexing particular logs

ed07net_YG — Wed, 12 Jan 2022 23:37:25 GMT

Hi community ,

I am trying to filter out some undesired traffic from a particular index. I read about the option using props.conf and transforms.conf.

The query matching the traffic that I don't want, looks like this:

index=abc sourcetype=abc_traffic dest_ip=255.255.255.255

The index abc is located in the search App.

So, I went to my Search Head -> opt/splunk/etc/apps/search/local and modified the props.conf with the following:

[abc_traffic]
TRANSFORMS-null= broadcast-null

Then, I created a TRANSFORMS.conf file in the same directory with the following entry:

[broadcast-null]
REGEX= dest_ip= 255.255.255.255
DEST_KEY= queue
FORMAT= nullQueue

Restarted splunk

I am not sure if I am doing something wrong, maybe I am using the wrong location or format, not sure, I don't have too much experience managing Splunk.

Appreciated any help!

Re: Stop indexing particular logs

richgalloway — Thu, 13 Jan 2022 01:14:32 GMT

Those setting go on the indexer(s) and/or heavy forwarder, whichever the data passes through first. Also, they have no effect on data already indexed. Only new events will be filtered.

Re: Stop indexing particular logs

ed07net_YG — Thu, 13 Jan 2022 02:44:39 GMT

Thanks for the idea.

However, we are sending those logs to a syslogng and then to our indexers.

So, I realized that I should have create the props.conf and transforms.conf in the opt/splunk/etc/system/local. However, same result, I am still getting the logs.

I believe that I might be choosing the wrong directory?

Thanks,

Re: Stop indexing particular logs

SinghK — Thu, 13 Jan 2022 04:42:08 GMT

no not exactly, if there is an addon that is used for that data like any TA that you are using this props and transforms will go into that TA.

you need to check which addon it is and add the data there and it will definitely be on your heavy forwarder/indexer you need to make changes a both locations for this to take affect as explained above @richgalloway .

Re: Stop indexing particular logs

richgalloway — Thu, 13 Jan 2022 12:57:42 GMT

Which app the file is in matters less then which server it's on. The settings MUST be on the indexer rather than the search head.

Re: Stop indexing particular logs

ed07net_YG — Thu, 13 Jan 2022 17:51:24 GMT

Thanks that helped,

At the end that was part of the issue and also that I was using the wrong sourcetype, looks like the Add-on was changing the sourcetype name. The following post helped me as well.

https://community.splunk.com/t5/Splunk-Search/Easiest-way-to-exclude-ingestion-of-events-for-a-specific-IP/m-p/289350