All Apps and Add-ons

send message when alert is passed

sbsbb
Builder

I'm able to send an email, when a condition is met, and also to setup the next time the alert has to be exectuted after the first match... But is it possible to send an email to inform that the alert is ok (values are normal again...) ?

0 Karma

yannK
Splunk Employee
Splunk Employee

add the condition in the search, instead of the alert condition.

mysearch | eval alert_result=if(condition=true,"status_alert","status_ok")

and send the alert every time with "always" , then you do the triage based on this field "alert_result".

sbsbb
Builder

What I missing in Splunk with your query, is a second condition for throttling, so I could configure :
alert condition : alert_result=true -> send alert email
execute search every X Minutes, until condition2 is met (alert_result=status_ok), and send email "alert ok"...

0 Karma

sbsbb
Builder

That's an interesting solution, but I that case, I get an alert everytime the search is run.
A proper alerting would be something like that

When there is no error, run the search once per hour

If alert is raised then
- send a alert message
- rerun the query maybe more often (every 15min)
- send a message when it is ok again, and run query with normal interval again...

0 Karma

sbsbb
Builder

Thank you for your proposition.
I really need an "ok email", because all our other alerting (outside from splunk) work like this.

I've understood you workaround, and I guess it will be working, it only a lot of handwork to make 2 alerts per query...

I found a duplicate, were it is described. Many thanks.
I hope Splunk will add such a feature in there wishlist for next release.

0 Karma

DaveSavage
Builder

sbsbb - we did this in a different way using python code to check a table on the 'shall I notify question' e.g. do I already know about it (a question you didn't ask, I guess, but could be an issue)..but all of that was outside of Splunk's email function. To keep it inside there is no reason why you couldn't create a lookup to see if a) you already know about it...and b) figure that people may like to know its ok now. A little more work on the notification qualification, but may achieve what you seek?
I'm not posting as an answer, because its just a potential , others may have something.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

SOC4Kafka - New Kafka Connector Powered by OpenTelemetry

The new SOC4Kafka connector, built on OpenTelemetry, enables the collection of Kafka messages and forwards ...

Rounding off the Splunk Dashboard Contest

What does a contest-winning Splunk dashboard look like? In this case, it isn't in a browser tab at all. It ...