topic Re: REGEX transforms.conf, NOT operator in Splunk Search

REGEX transforms.conf, NOT operator

aspa — Mon, 07 May 2012 14:25:58 GMT

Hi Splunkers

I'm new to splunk and currently playing around with the heavy forwarder. I found here several examples how to match a specific string in an event and only forward that event to the indexer. However, I could not find the opposite.

Here is what I would like to do.

Let's say in /var/log/messages we have those two log entries:

May 7 08:00:14 <host> <user>: Splunk, get that line! May 7 08:01:45 <host> <user>: Splunk, miss that line!

My props.conf contains:

[source::/var/log/messages] TRANSFORMS-set=setnull,ignore

My transforms.conf contains:

[setnull] REGEX = . DEST_KEY = queue FORMAT = nullQueue

[ignore] REGEX = !miss DEST_KEY = queue FORMAT = indexQueue

I have tryed several regex for the NOT miss part. So far I have not found any solution. Again, my goal is not to match the first line. I know how that is done. I want to match everything except something with a key word. At the end I should be able to exclude known events and always receive on my indexer what is wanted or sofar unknown.

Thank you for your help

Re: REGEX transforms.conf, NOT operator

kristian_kolb — Mon, 28 Sep 2020 11:46:38 GMT

Have you looked at the docs?

If you have a known event that you do NOT want, you can use a matching REGEX to send that event to the nullQueue. The rest are automatically sent to the parsingQueue.

http://docs.splunk.com/Documentation/Splunk/4.3.1/Deploy/Routeandfilterdatad#Discard_specific_events_and_keep_the_rest

If you want to only keep a known set of events, you do as you did above, i.e. set the queue to nullQueue for all events, and then overwrite that with parsingQueue for the events you wish to keep.

http://docs.splunk.com/Documentation/Splunk/4.3.1/Deploy/Routeandfilterdatad#Keep_specific_events_and_discard_the_rest

UPDATE:
Saw that you send the events to indexQueue. You might be right but you probably want the parsingQueue.
Hope this helps,

Kristian

Re: REGEX transforms.conf, NOT operator

aspa — Tue, 08 May 2012 12:35:35 GMT

Hello Kristian

Thank you for your reply. I figured out where my failure was in the configuration. Your post helped me a lot!

I just post shortly my solution:

The props.conf remains almost the same as above:

[source::/var/log/messages] TRANSFORMS-set=ignore

The transforms.conf changed the most and is now shorter:

[ignore] REGEX = miss DEST_KEY = queue FORMAT = nullQueue

By sending the event matching the REGEX to the nullQueue, the event is not forwarded to the indexer.