Getting Data In

How to drop events on a heavy forwarder?

vinaypradhan
Explorer

hi, 
I have a heavy forwarder configured this way

inputs.conf
[udp://514]
sourcetype = syslog
index = abc
connection_host = ip
disabled = 0
_TCP_ROUTING =MyRoute

So any server sending syslog on UDP 514 is hitting my heavy forwarder and getting sent to my indexers. 
I want to block events from a set of servers, as I don't want them to be indexed. They are consuming my licenses. 
I am trying to do this in props.conf and transforms.conf

props.conf
[host::XX.XX.XXX.XXX]
TRANSFORMS-abc = drop_events

and in my transforms.conf, I have this 

[drop_events]
REGEX =.
DEST_KEY = queue
FORMAT = nullQueue

But this is not dropping the events from this server. I can still events being indexed real time when i search. 

What am I doing wrong? I have a few more servers which I need to block this way. 

Any help is greatly appreciated. 


Labels (1)
0 Karma
1 Solution

vinaypradhan
Explorer

For syslog events, dropping events works in a different way. I was able to finally get it to work. Posting it here, so that it might help someone else who is facing the same kind of situation. 

Because of the way Splunk identifies and cooks a syslog, the syslog event ends up getting cooked using one of the configurations in the default props.conf called syslog-host
We will have to override that so that the events get dropped first before Splunk tries to cook the event. 

For example, if we want to drop all syslog events coming from server that have expr in their names, then this needs to be done this way

Ad this to prop.sconf

#syslog
[syslog]
TRANSFORMS = drop_events_syslog, syslog-host

And add drop_events_syslog in transforms.conf:

[drop_events_syslog]
REGEX = :\d\d\s+(?:\d+\s+|(?:user|daemon|local.?)\.\w+\s+)*\[?(.*expr.*)\]?\s
DEST_KEY = queue
FORMAT = nullQueue

The most important thing here is that, in props.conf, we have to mention our drop stanza first and then also mention the syslog-host stanza. This will ensure syslog-host stanza gets handled after drop gets handled.

View solution in original post

vinaypradhan
Explorer

For syslog events, dropping events works in a different way. I was able to finally get it to work. Posting it here, so that it might help someone else who is facing the same kind of situation. 

Because of the way Splunk identifies and cooks a syslog, the syslog event ends up getting cooked using one of the configurations in the default props.conf called syslog-host
We will have to override that so that the events get dropped first before Splunk tries to cook the event. 

For example, if we want to drop all syslog events coming from server that have expr in their names, then this needs to be done this way

Ad this to prop.sconf

#syslog
[syslog]
TRANSFORMS = drop_events_syslog, syslog-host

And add drop_events_syslog in transforms.conf:

[drop_events_syslog]
REGEX = :\d\d\s+(?:\d+\s+|(?:user|daemon|local.?)\.\w+\s+)*\[?(.*expr.*)\]?\s
DEST_KEY = queue
FORMAT = nullQueue

The most important thing here is that, in props.conf, we have to mention our drop stanza first and then also mention the syslog-host stanza. This will ensure syslog-host stanza gets handled after drop gets handled.

vinaypradhan
Explorer

I even tried changing [host::XX.XX.XXX.XXX] to [host::hostname] but still no luck

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...