Splunk Search

How to ignore some events at index time?

ktn01
Path Finder

Hello,
I have to index only events that contains the string "$$log$$".
I try with a transforms like

[ignore]
REGEX = < a regex to match event not containing "$$log$$" >
DEST_KEY = queue
FORMAT = nullQueue

How can I wrote a "not match" regex (like !~ in perl)?

Thank's
Christian

0 Karma
1 Solution

somesoni2
Revered Legend

Instead of using the 'not match' regex and drop those events, you do 'match' regex and index those matching events, and drop everything else. Something on the lines of

In props.conf (keep the exact same order of transforms):

[source::/var/log/messages]
TRANSFORMS-set= setnull,setparsing

In transforms.conf:

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = \$\$log\$\$
DEST_KEY = queue
FORMAT = indexQueue

See this for more details.

http://docs.splunk.com/Documentation/Splunk/6.5.2/Forwarding/Routeandfilterdatad#Keep_specific_event...

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

Regex doesn't do NOT very well. Your best bet is write a regex that matches want you want to keep and send everything else to nullQueue. See the answer at https://answers.splunk.com/answers/111362/filtering-of-events-using-nullqueue.html.

---
If this reply helps you, Karma would be appreciated.

somesoni2
Revered Legend

Instead of using the 'not match' regex and drop those events, you do 'match' regex and index those matching events, and drop everything else. Something on the lines of

In props.conf (keep the exact same order of transforms):

[source::/var/log/messages]
TRANSFORMS-set= setnull,setparsing

In transforms.conf:

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = \$\$log\$\$
DEST_KEY = queue
FORMAT = indexQueue

See this for more details.

http://docs.splunk.com/Documentation/Splunk/6.5.2/Forwarding/Routeandfilterdatad#Keep_specific_event...

Get Updates on the Splunk Community!

Developer Spotlight with William Searle

The Splunk Guy: A Developer’s Path from Web to Cloud William is a Splunk Professional Services Consultant with ...

Major Splunk Upgrade – Prepare your Environment for Splunk 10 Now!

Attention App Developers: Test Your Apps with the Splunk 10.0 Beta and Ensure Compatibility Before the ...

Stay Connected: Your Guide to June Tech Talks, Office Hours, and Webinars!

What are Community Office Hours?Community Office Hours is an interactive 60-minute Zoom series where ...