topic Re: Discarding Certain Events in Monitoring Splunk

Discarding Certain Events

Gorwinn — Wed, 08 Jan 2025 17:27:41 GMT

Hello All!

I am trying to discard a certain event before the Indexers Ingest it using keyword envoy. Below is an example

timestamp vcenter envoy-access 2024-12-29T23:53:56.632Z info envoy[139855859431232] [Originator@6876 sub=Default] 2024-12-29T23:53:50.392Z POST /sdk HTTP/1.1 200 via_upstream

I tried creating props and transforms conf in $SPLUNK_HOME/etc/system/local but it's not working. My questions are if my stanzas are correct and if I should put them in local directory? Appreciate any assistance you can provide, Thank you.

Props.conf [nullQueue] queue = nullQueue [host::vcenter] TRANSFORMS-null = setnull [source::/var/log/remote/catchall/(IPAddress of Vcenter)/*.log] TRANSFORMS-null = setnull transforms.conf [setnull] REGEX = envoy DEST_KEY = queue FORMAT = nullQueue

Re: Discarding Certain Events

gcusello — Wed, 08 Jan 2025 16:40:21 GMT

Hi @Gorwinn ,

let me understand, you want to take all the exents except the ones containing the word "envoy", is it correct?

at first, how are you taking these logs?

if using an Heavy Forwarder, you have to put the props.conf and transforms.conf on the first Splunk Full instance that data pass trhough, in other words on the Heavy Forwarder, if present or on the Indexer.

then, the transformation names must me unique in props.conf:

[host::vcenter] TRANSFORMS-null = setnull [source::/var/log/remote/catchall/*/*.log] TRANSFORMS-null2 = setnull

then check the regex using the rex command in Splunk.

Anyway, the issue usually is the location of the conf files (obviously I suppose that you restarted Splunk after conf files modification!).

The documentation is at https://docs.splunk.com/Documentation/Splunk/9.4.0/Forwarding/Routeandfilterdatad#Filter_event_data_and_send_to_queues

Ciao.

Giuseppe

Re: Discarding Certain Events

richgalloway — Wed, 08 Jan 2025 16:46:26 GMT

The queue = nullQueue setting is not valid in props.conf.

Make sure the host and source names match that of the incoming data. Consider adding a sourcetype stanza for the data.

The stanzas belong in the first full instance of Splunk that processes the data (indexers and HFs). Put them in the default directory of a custom app.

Re: Discarding Certain Events

Gorwinn — Wed, 08 Jan 2025 19:10:07 GMT

Yes, I want to take all logs but events with envoy in it. I am only using universal forwarder which I believe cant parse any data like a heavy forwarder. Am I mistaken? I made the transformation names unique and restarted splunk via GUI but still no discards.

Re: Discarding Certain Events

isoutamo — Wed, 08 Jan 2025 20:06:12 GMT

One old post which do exactly what you are doing.
https://community.splunk.com/t5/Getting-Data-In/sending-specific-events-to-nullqueue-using-props-amp-amp/m-p/660688