topic Re: How to drop events on a heavy forwarder? in Getting Data In

How to drop events on a heavy forwarder?

vinaypradhan — Wed, 31 Mar 2021 20:55:54 GMT

hi,
I have a heavy forwarder configured this way

inputs.conf
[udp://514]
sourcetype = syslog
index = abc
connection_host = ip
disabled = 0
_TCP_ROUTING =MyRoute

So any server sending syslog on UDP 514 is hitting my heavy forwarder and getting sent to my indexers.
I want to block events from a set of servers, as I don't want them to be indexed. They are consuming my licenses.
I am trying to do this in props.conf and transforms.conf

props.conf
[host::XX.XX.XXX.XXX]
TRANSFORMS-abc = drop_events

and in my transforms.conf, I have this

[drop_events]
REGEX =.
DEST_KEY = queue
FORMAT = nullQueue

But this is not dropping the events from this server. I can still events being indexed real time when i search.

What am I doing wrong? I have a few more servers which I need to block this way.

Any help is greatly appreciated.

Re: How to drop events on a heavy forwarder?

vinaypradhan — Wed, 31 Mar 2021 21:42:27 GMT

I even tried changing [host::XX.XX.XXX.XXX] to [host::hostname] but still no luck

Re: How to drop events on a heavy forwarder?

vinaypradhan — Thu, 29 Jul 2021 15:52:22 GMT

For syslog events, dropping events works in a different way. I was able to finally get it to work. Posting it here, so that it might help someone else who is facing the same kind of situation.

Because of the way Splunk identifies and cooks a syslog, the syslog event ends up getting cooked using one of the configurations in the default props.conf called syslog-host
We will have to override that so that the events get dropped first before Splunk tries to cook the event.

For example, if we want to drop all syslog events coming from server that have expr in their names, then this needs to be done this way

Ad this to prop.sconf

#syslog
[syslog]
TRANSFORMS = drop_events_syslog, syslog-host

And add drop_events_syslog in transforms.conf:

[drop_events_syslog]
REGEX = :\d\d\s+(?:\d+\s+|(?:user|daemon|local.?)\.\w+\s+)*\[?(.*expr.*)\]?\s
DEST_KEY = queue
FORMAT = nullQueue

The most important thing here is that, in props.conf, we have to mention our drop stanza first and then also mention the syslog-host stanza. This will ensure syslog-host stanza gets handled after drop gets handled.