Alerting

How to set up a email notification for an alert with the output of another search?

bsuresh1
Path Finder

I want to set up an alert with
-> a real time search or a scheduled search
-> check the string "fatal error" (eg., 2017-02-08 05:34:56, 456 [Thread=12345]: << 400 HTTP request failed: fatal error occurred)
-> if above string is found, search for the string "Thread=12345"(this string number will be different for each event) for last two days and populate that as results in Email notification

Is this is possible? If yes, please guide me the steps to do. Thanks.

0 Karma

mirkoneverstops
Path Finder

Option A: using |join

index=myindex sourcetype=mysourcetype "fatal error" 
|rex field=_raw "\d\s\[(?<th_string>\S+)\]:\s"
|join th_string[search earliest=-2d latest=now index=myindex sourcetype=mysourcetype]

Option B: using lookups
First scheduled search:

index=myindex sourcetype=mysourcetype "fatal error" 
|rex field=_raw "\d\s\[(?<th_string>\S+)\]:\s"|outputlookup create_empty=true fatal_error_th_list.csv

Second scheduled search:

|inputlookup fatal_error_th_list.csv|append [search earliest=-2d latest=now index=myindex sourcetype=mysourcetype]

In both cases remember subsearch limit (should be 50,000 rows).

0 Karma
Get Updates on the Splunk Community!

.conf25 technical session recap of Observability for Gen AI: Monitoring LLM ...

If you’re unfamiliar, .conf is Splunk’s premier event where the Splunk community, customers, partners, and ...

A Season of Skills: New Splunk Courses to Light Up Your Learning Journey

There’s something special about this time of year—maybe it’s the glow of the holidays, maybe it’s the ...

Announcing the Migration of the Splunk Add-on for Microsoft Azure Inputs to ...

Announcing the Migration of the Splunk Add-on for Microsoft Azure Inputs to Officially Supported Splunk ...