Getting Data In

Excluding Specific keywords - Heavy Forwarder

Communicator

I'm wanting to exclude records with a particular keyword from being ingested by the indexer.

I have several Windows servers all pointing to a heavy forwarder where the inputs.conf file determines which logs to ingest into the Splunk indexer however there is some selected content that I want to exclude that exists in some of the included logs.

Specifically, I want to exclude any records that contain the word "Zabbix", or "Zabbix Agent".

How can this be done and where is the best place to do this filtering?

0 Karma
1 Solution

Contributor

Hi dyude @balcv ,

You can write a props and transforms for this ...

props.conf

[Your sourcetype]
TRANSFORMS-set= zabbix,zabbix_agent

transforms.conf

[zabbix]
REGEX = Zabbix
DEST_KEY = queue
FORMAT = nullQueue

[zabbix_agent]
REGEX = Zabbix\sAgent
DEST_KEY = queue
FORMAT = nullQueue

This will exclude all the Zabbix and Zabbix Agent keywords present in the logs.

Try this out and let me know if it works for you!

View solution in original post

Contributor

Hi dyude @balcv ,

You can write a props and transforms for this ...

props.conf

[Your sourcetype]
TRANSFORMS-set= zabbix,zabbix_agent

transforms.conf

[zabbix]
REGEX = Zabbix
DEST_KEY = queue
FORMAT = nullQueue

[zabbix_agent]
REGEX = Zabbix\sAgent
DEST_KEY = queue
FORMAT = nullQueue

This will exclude all the Zabbix and Zabbix Agent keywords present in the logs.

Try this out and let me know if it works for you!

View solution in original post

Communicator

It looks like the config details provided by vinod94 were in fact correct however I needed to modify the props.conf and transforms.conf on the indexer box and NOT on the heavy forwarder.

When I worked through the data flow, the heavy forwarder is only being used as the deployment server and not receiving the logs for these specific data sources. Once I updated the files on the indexer, I got the exact results I was hoping for.

Thank you.

Contributor

Glad! it worked for you! (Y)

0 Karma

Communicator

Thanks for the details @vindod94 . Much appreciated.

One question, in the props.conf, you have [Your sourcetype]. What should be in this header? Does it relate to a windows log or is it just a name I assign it?

Thanks

0 Karma

Contributor

@balcv,

You just have to put Your Sourcetype Name for which you are filtering the logs . So basically you can do this for a host OR source OR sourcetype.

You can follow this props.conf doc, this will give you an idea.

https://docs.splunk.com/Documentation/Splunk/7.2.4/Admin/Propsconf

0 Karma

Communicator

Thanks very much. I have added the code as suggested, and restarted the heavy forwarder, however the Zabbix items are still getting through to the indexer.

0 Karma

Contributor

@balcv,
Have you applied it on your sourcetype(your sourcetype name)?

0 Karma

Communicator

I think so, yes.

Props.conf
[source::WinEventLog:Application]
TRANSFORMS-set= zabbix

Data according to indexer:

index="winEventLog"

3/22/19 8:22:52.000 AM 03/22/2019
08:22:52 AM LogName=Application
SourceName=Zabbix Agent EventCode=1
EventType=3 Show all 19 lines host
= EXIGE source = WinEventLog:Application

Does this look correct?

0 Karma