Splunk Search

How to ignore some events at index time?

ktn01
Path Finder

Hello,
I have to index only events that contains the string "$$log$$".
I try with a transforms like

[ignore]
REGEX = < a regex to match event not containing "$$log$$" >
DEST_KEY = queue
FORMAT = nullQueue

How can I wrote a "not match" regex (like !~ in perl)?

Thank's
Christian

0 Karma
1 Solution

somesoni2
Revered Legend

Instead of using the 'not match' regex and drop those events, you do 'match' regex and index those matching events, and drop everything else. Something on the lines of

In props.conf (keep the exact same order of transforms):

[source::/var/log/messages]
TRANSFORMS-set= setnull,setparsing

In transforms.conf:

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = \$\$log\$\$
DEST_KEY = queue
FORMAT = indexQueue

See this for more details.

http://docs.splunk.com/Documentation/Splunk/6.5.2/Forwarding/Routeandfilterdatad#Keep_specific_event...

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

Regex doesn't do NOT very well. Your best bet is write a regex that matches want you want to keep and send everything else to nullQueue. See the answer at https://answers.splunk.com/answers/111362/filtering-of-events-using-nullqueue.html.

---
If this reply helps you, Karma would be appreciated.

somesoni2
Revered Legend

Instead of using the 'not match' regex and drop those events, you do 'match' regex and index those matching events, and drop everything else. Something on the lines of

In props.conf (keep the exact same order of transforms):

[source::/var/log/messages]
TRANSFORMS-set= setnull,setparsing

In transforms.conf:

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = \$\$log\$\$
DEST_KEY = queue
FORMAT = indexQueue

See this for more details.

http://docs.splunk.com/Documentation/Splunk/6.5.2/Forwarding/Routeandfilterdatad#Keep_specific_event...

Get Updates on the Splunk Community!

Now Playing: Splunk Education Summer Learning Premieres

It’s premiere season, and Splunk Education is rolling out new releases you won’t want to miss. Whether you’re ...

The Visibility Gap: Hybrid Networks and IT Services

The most forward thinking enterprises among us see their network as much more than infrastructure – it's their ...

Get Operational Insights Quickly with Natural Language on the Splunk Platform

In today’s fast-paced digital world, turning data into actionable insights is essential for success. With ...