How to set up a email notification for an alert with the output of another search?


I want to set up an alert with
-> a real time search or a scheduled search
-> check the string "fatal error" (eg., 2017-02-08 05:34:56, 456 [Thread=12345]: << 400 HTTP request failed: fatal error occurred)
-> if above string is found, search for the string "Thread=12345"(this string number will be different for each event) for last two days and populate that as results in Email notification

Is this is possible? If yes, please guide me the steps to do. Thanks.

0 Karma

Path Finder

Option A: using |join

index=myindex sourcetype=mysourcetype "fatal error" 
|rex field=_raw "\d\s\[(?<th_string>\S+)\]:\s"
|join th_string[search earliest=-2d latest=now index=myindex sourcetype=mysourcetype]

Option B: using lookups
First scheduled search:

index=myindex sourcetype=mysourcetype "fatal error" 
|rex field=_raw "\d\s\[(?<th_string>\S+)\]:\s"|outputlookup create_empty=true fatal_error_th_list.csv

Second scheduled search:

|inputlookup fatal_error_th_list.csv|append [search earliest=-2d latest=now index=myindex sourcetype=mysourcetype]

In both cases remember subsearch limit (should be 50,000 rows).

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!