Alerting

Set an Alert to Throttle until the end of the current day

x213217
Explorer

I have a search that i only want to have triggered once a day (if it gets a result of course). The issue is that i have to run the alert every 30 minutes within certain hours of the day and if i set a hard-coded throttle it will affect the next days window. And if i dont set a throttle I will get a spam of alerts every 30 mins.

I would like to have the alert's throttle reset at 00:00 each day.

here is my search , i simply look for a success event for a job, and generates a result

index= sourcetype=
jobName=theJobName
status="SUCCESS"
| eval endTimeEpoch = strptime(endTime, "%Y-%m-%d %H:%M:%S") 
| eval endTime = strftime(endTimeEpoch, "%B %d %Y %H:%M:%S")
| eval Platform=case(like(jobName,"jobName here"), "platform name here"
)
| table Platform jobName status endTime
0 Karma
1 Solution

harsmarvania57
Ultra Champion

Hi ,

You can try below query

index=<yourindex> sourcetype=<yoursourcetype>
jobName=theJobName
status="SUCCESS"
| eval endTimeEpoch = strptime(endTime, "%Y-%m-%d %H:%M:%S") 
| eval endTime = strftime(endTimeEpoch, "%B %d %Y %H:%M:%S")
| eval Platform=case(like(jobName,"jobName here"), "platform name here")
| eval date=strftime(_time, "%d/%m/%Y")
| table Platform jobName status endTime date

Now while creating Alert, please use below settings

1.) Trigger Conditions -> Trigger when Number of results is greater than 0
2.) Trigger -> For each result
3.) Enable Throttle
4.) Suppress result containing field value -> date
5.) Suppress triggering for -> 24 hours

With above settings if your search is generating more than one result then splunk will trigger email alert for each result first time but it will suppress alert for rest of the day because date field value will be same for rest of the day.

View solution in original post

0 Karma

harsmarvania57
Ultra Champion

Hi ,

You can try below query

index=<yourindex> sourcetype=<yoursourcetype>
jobName=theJobName
status="SUCCESS"
| eval endTimeEpoch = strptime(endTime, "%Y-%m-%d %H:%M:%S") 
| eval endTime = strftime(endTimeEpoch, "%B %d %Y %H:%M:%S")
| eval Platform=case(like(jobName,"jobName here"), "platform name here")
| eval date=strftime(_time, "%d/%m/%Y")
| table Platform jobName status endTime date

Now while creating Alert, please use below settings

1.) Trigger Conditions -> Trigger when Number of results is greater than 0
2.) Trigger -> For each result
3.) Enable Throttle
4.) Suppress result containing field value -> date
5.) Suppress triggering for -> 24 hours

With above settings if your search is generating more than one result then splunk will trigger email alert for each result first time but it will suppress alert for rest of the day because date field value will be same for rest of the day.

0 Karma
Get Updates on the Splunk Community!

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...

New! Splunk Observability Search Enhancements for Splunk APM Services/Traces and ...

Regardless of where you are in Splunk Observability, you can search for relevant APM targets including service ...

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...