Alerting

How to set up alerts which search for every minute and send out alerts if response time is greater than 1000 msec?

rahul_n
Explorer

Hi. 

I am trying to set up alerts to notify when the response time is greater than 1000 milli seconds. The alert has to search for every minute or for every 5 minutes. 

Below is the query which I have used.

index=testIndex sourcetype=testSourceType basicQuery
| where executionTime>1000
| stats count by app_name, executionTime

After running the query by setting it for "Last 5 minutes" in the dropdown beside search icon, I am getting results. Then I have saved the query as an Alert  with time range set to "last 5 minutes" and Cron Expression set to "*/1 * * * *" to run it for every 1 minute in the last 5 minutes. 

Is this a correct approach ? Main point is: I don't want to miss any events with response time more than 1000msec. 

 

Also, what is the difference between setting time in dropdown and  earliest=-5m latest=now ? 

Can someone please help me ? 

 

Thanks in Advance. 

Labels (5)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

That is a valid approach.  However, by looking back 5 minutes every minute you run the risk of seeing (and alerting on) the same event 5 times.  If you're concerned about delays in events getting indexed then give them time to do so.

index=testIndex sourcetype=testSourceType basicQuery earliest=-2m latest=-1m
...

The difference between setting time in the dropdown and setting it in the query is the query overrides the dropdown.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Splunk Decoded: Service Maps vs Service Analyzer Tree View vs Flow Maps

It’s Monday morning, and your phone is buzzing with alert escalations – your customer-facing portal is running ...

What’s New in Splunk Observability – September 2025

What's NewWe are excited to announce the latest enhancements to Splunk Observability, designed to help ITOps ...

Fun with Regular Expression - multiples of nine

Fun with Regular Expression - multiples of nineThis challenge was first posted on Slack #regex channel ...