Alerting

Report sent after Alert is triggered - is it possible?

tengugurl
New Member

Hi there!
New user here, I am looking to simplify our troubleshooting work here at work by doing the following:

1) When an Alert is triggered (Regardless of the reason/search parameters)

2) A subsequent report will be sent after the Alert is triggered. (AKA the Search parameters one would be looking to use to better investigate the alert) 

Is this possible?

Labels (1)
0 Karma

anilchaithu
Builder

@tengugurl 

Its quite possible to sent the search parameters (caused alert) to users by choosing "send email" alert action. You can always include job fields as tokens in the email.

This is better option.

anilchaithu_0-1594434394015.png

If you don't want to use this alert actions, you can create savedsearch to search REST end point to get the results of triggered alerts and send an email with the result set.

| rest /services/alerts/fired_alerts

 

Hope this helps

0 Karma

tengugurl
New Member

Hi @anilchaithu 
thanks for the thoughtful reply.
I love this idea, and have currently been adding the search parameters to the email action but was thinking of adding a separate search parameter as a follow up.

E.g Alert triggered (Your service is unfire!) separate report triggers that shows the HTTP status codes for the past hour. (Just to see how unfire it really is in comparison)

hmm I am curious on the second option you provided:

| rest /services/alerts/fired_alerts

Does this sound offbase?

Imagine I made a search that was:
index=ABC source=X |timechart count by status
then added 
| rest/services/alerts/named_alert

E.g 

index=ABC source=X |timechart count by status|rest/services/alerts/named_alert

 

I would get the email with the timechart of status when "named_alert" triggers?

 

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Thanks for the Memories: .conf26 Took Learning to New Heights

Thank you, Splunk Community, for making .conf26 in Denver one for the books. From packed Splunk University ...

Best Practices: Splunk auto adjust pipeline queue

When you enable autoAdjustQueue in Splunk, maxSize should be understood as the queue size Splunk starts with ...

Splunk Auto Ingestion Parallel Pipeline Scaling

Why this feature matters Many Splunk environments experience ingestion pressure long before the host is fully ...