Splunk Search

How to get alerts when event X isn't followed by event Y within a certain length of time?

andrew_burnett
Path Finder

I need an alert where you get this message "Attempting to send email to:<email>" but you don't ever get the message "Email sent successfully" within 1-2 minutes. The problem is if you don't get the second message, a restart will be required before you do. So I need event 1, but no event 2

Labels (1)
Tags (2)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @andrew_burnett,

you should try something like this:

index=your_index ("Attempting to send email to:" OR "Email sent successfully")
| eval message_kind=if(searchmatch("Attempting to send email to:"),"1","2")
| stats dc(message_kind) AS dc_message_kind values(message_kind) AS message_kind
| where dc_message_kind=1 AND message_kind="1"

Ciao.

Giuseppe

0 Karma

andrew_burnett
Path Finder

My question about it is, when the "Attempting to send email to:" gets stuck, all the logs stop. Would that search work in this case?

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @andrew_burnett,

as you asked the above search takes logs and check if the first message is followed by the second, the alert triggers when the first message isn't followed by the second (the where condition).

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...