topic Re: Excluding Specific keywords - Heavy Forwarder in Getting Data In

Excluding Specific keywords - Heavy Forwarder

balcv — Wed, 13 Mar 2019 05:03:41 GMT

I'm wanting to exclude records with a particular keyword from being ingested by the indexer.

I have several Windows servers all pointing to a heavy forwarder where the inputs.conf file determines which logs to ingest into the Splunk indexer however there is some selected content that I want to exclude that exists in some of the included logs.

Specifically, I want to exclude any records that contain the word "Zabbix", or "Zabbix Agent".

How can this be done and where is the best place to do this filtering?

Re: Excluding Specific keywords - Heavy Forwarder

vinod94 — Wed, 13 Mar 2019 06:26:49 GMT

Hi dyude @balcv ,

You can write a props and transforms for this ...

props.conf

[Your sourcetype]
TRANSFORMS-set= zabbix,zabbix_agent

transforms.conf

[zabbix]
REGEX = Zabbix
DEST_KEY = queue
FORMAT = nullQueue

[zabbix_agent]
REGEX = Zabbix\sAgent
DEST_KEY = queue
FORMAT = nullQueue

This will exclude all the Zabbix and Zabbix Agent keywords present in the logs.

Try this out and let me know if it works for you!

Re: Excluding Specific keywords - Heavy Forwarder

balcv — Wed, 13 Mar 2019 21:37:07 GMT

Thanks for the details @vindod94 . Much appreciated.

One question, in the props.conf, you have [Your sourcetype]. What should be in this header? Does it relate to a windows log or is it just a name I assign it?

Thanks

Re: Excluding Specific keywords - Heavy Forwarder

vinod94 — Thu, 14 Mar 2019 05:45:28 GMT

@balcv,

You just have to put Your Sourcetype Name for which you are filtering the logs . So basically you can do this for a host OR source OR sourcetype.

You can follow this props.conf doc, this will give you an idea.

https://docs.splunk.com/Documentation/Splunk/7.2.4/Admin/Propsconf

Re: Excluding Specific keywords - Heavy Forwarder

balcv — Thu, 21 Mar 2019 03:47:05 GMT

Thanks very much. I have added the code as suggested, and restarted the heavy forwarder, however the Zabbix items are still getting through to the indexer.

Re: Excluding Specific keywords - Heavy Forwarder

vinod94 — Thu, 21 Mar 2019 08:19:01 GMT

@balcv,
Have you applied it on your sourcetype(your sourcetype name)?

Re: Excluding Specific keywords - Heavy Forwarder

balcv — Thu, 21 Mar 2019 21:24:56 GMT

I think so, yes.

Props.conf
[source::WinEventLog:Application]
TRANSFORMS-set= zabbix

Data according to indexer:

index="winEventLog"

3/22/19 8:22:52.000 AM 03/22/2019
08:22:52 AM LogName=Application
SourceName=Zabbix Agent EventCode=1
EventType=3 Show all 19 lines host
= EXIGE source = WinEventLog:Application

Does this look correct?

Re: Excluding Specific keywords - Heavy Forwarder

balcv — Mon, 25 Mar 2019 01:43:54 GMT

It looks like the config details provided by vinod94 were in fact correct however I needed to modify the props.conf and transforms.conf on the indexer box and NOT on the heavy forwarder.

When I worked through the data flow, the heavy forwarder is only being used as the deployment server and not receiving the logs for these specific data sources. Once I updated the files on the indexer, I got the exact results I was hoping for.

Thank you.

Re: Excluding Specific keywords - Heavy Forwarder

vinod94 — Mon, 25 Mar 2019 04:46:40 GMT

Glad! it worked for you! (Y)