Splunk Search

How to ignore some events at index time?

ktn01
Path Finder

Hello,
I have to index only events that contains the string "$$log$$".
I try with a transforms like

[ignore]
REGEX = < a regex to match event not containing "$$log$$" >
DEST_KEY = queue
FORMAT = nullQueue

How can I wrote a "not match" regex (like !~ in perl)?

Thank's
Christian

0 Karma
1 Solution

somesoni2
Revered Legend

Instead of using the 'not match' regex and drop those events, you do 'match' regex and index those matching events, and drop everything else. Something on the lines of

In props.conf (keep the exact same order of transforms):

[source::/var/log/messages]
TRANSFORMS-set= setnull,setparsing

In transforms.conf:

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = \$\$log\$\$
DEST_KEY = queue
FORMAT = indexQueue

See this for more details.

http://docs.splunk.com/Documentation/Splunk/6.5.2/Forwarding/Routeandfilterdatad#Keep_specific_event...

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

Regex doesn't do NOT very well. Your best bet is write a regex that matches want you want to keep and send everything else to nullQueue. See the answer at https://answers.splunk.com/answers/111362/filtering-of-events-using-nullqueue.html.

---
If this reply helps you, Karma would be appreciated.

somesoni2
Revered Legend

Instead of using the 'not match' regex and drop those events, you do 'match' regex and index those matching events, and drop everything else. Something on the lines of

In props.conf (keep the exact same order of transforms):

[source::/var/log/messages]
TRANSFORMS-set= setnull,setparsing

In transforms.conf:

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = \$\$log\$\$
DEST_KEY = queue
FORMAT = indexQueue

See this for more details.

http://docs.splunk.com/Documentation/Splunk/6.5.2/Forwarding/Routeandfilterdatad#Keep_specific_event...

Get Updates on the Splunk Community!

Observe and Secure All Apps with Splunk

  Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

Splunk Decoded: Business Transactions vs Business IQ

It’s the morning of Black Friday, and your e-commerce site is handling 10x normal traffic. Orders are flowing, ...

Fastest way to demo Observability

I’ve been having a lot of fun learning about Kubernetes and Observability. I set myself an interesting ...