Splunk Search

Keep specific events and discard the rest (PART 2)

Log_wrangler
Builder

Hi,

I followed previous instructions and successfully was able to keep only ERROR and WARN logs and "discard the rest" from my_app logs that I am monitoring with a UF and filtering on the HF.

Now I need to filter down the ERROR and WARN logs a bit more, e.g. remove any event that contains foo-failure.

Here is what I have (based on previous answer)

Props.conf

[source::/temp/my_app/logs/*]
TRANSFORMS-set=setnull_my_app_logs,setparsing_my_app_logs

Transforms.conf

[setnull_my_app_logs]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing_my_app_logs]
REGEX = (?i)(ERROR|WARN)
DEST_KEY = queue
FORMAT = indexQueue

Do I need to add another setnull stanza in transforms.conf ???

Like
[setnull_my_app_logs]
REGEX = foo-failure
DEST_KEY = queue
FORMAT = nullQueue

or do I need to integrate "foo-failure" into the discard stanza somehow?

Thank you

0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

See if this works for you

Props.conf

[source::/temp/my_app/logs/*]
TRANSFORMS-set=setnull_my_app_logs,setparsing_error_warn_logs,setnull_foo_error

Transforms.conf

[setnull_my_app_logs]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing_error_warn_logs]
REGEX = (?i)(ERROR|WARN)
DEST_KEY = queue
FORMAT = indexQueue

[setnull_foo_error]
REGEX = foo-failure
DEST_KEY = queue
FORMAT = nullQueue

So,setnull_foo_error marks all events to go to nullQueue, setparsing_error_warn_logs marks all ERROR/WARN events to indexQueue leaving rest to still marked to go to nullQueue, at last, setnull_foo_error marks all ERROR/WARN events which contains foo-failure to mark to go to nullQueue, thus only ERROR/WARN events which DO NOT contains foo-failure will get indexed.

View solution in original post

0 Karma

somesoni2
SplunkTrust
SplunkTrust

See if this works for you

Props.conf

[source::/temp/my_app/logs/*]
TRANSFORMS-set=setnull_my_app_logs,setparsing_error_warn_logs,setnull_foo_error

Transforms.conf

[setnull_my_app_logs]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing_error_warn_logs]
REGEX = (?i)(ERROR|WARN)
DEST_KEY = queue
FORMAT = indexQueue

[setnull_foo_error]
REGEX = foo-failure
DEST_KEY = queue
FORMAT = nullQueue

So,setnull_foo_error marks all events to go to nullQueue, setparsing_error_warn_logs marks all ERROR/WARN events to indexQueue leaving rest to still marked to go to nullQueue, at last, setnull_foo_error marks all ERROR/WARN events which contains foo-failure to mark to go to nullQueue, thus only ERROR/WARN events which DO NOT contains foo-failure will get indexed.

0 Karma

Log_wrangler
Builder

Thank you!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...