Alerting

How to set up alerts which search for every minute and send out alerts if response time is greater than 1000 msec?

rahul_n
Explorer

Hi. 

I am trying to set up alerts to notify when the response time is greater than 1000 milli seconds. The alert has to search for every minute or for every 5 minutes. 

Below is the query which I have used.

index=testIndex sourcetype=testSourceType basicQuery
| where executionTime>1000
| stats count by app_name, executionTime

After running the query by setting it for "Last 5 minutes" in the dropdown beside search icon, I am getting results. Then I have saved the query as an Alert  with time range set to "last 5 minutes" and Cron Expression set to "*/1 * * * *" to run it for every 1 minute in the last 5 minutes. 

Is this a correct approach ? Main point is: I don't want to miss any events with response time more than 1000msec. 

 

Also, what is the difference between setting time in dropdown and  earliest=-5m latest=now ? 

Can someone please help me ? 

 

Thanks in Advance. 

Labels (5)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

That is a valid approach.  However, by looking back 5 minutes every minute you run the risk of seeing (and alerting on) the same event 5 times.  If you're concerned about delays in events getting indexed then give them time to do so.

index=testIndex sourcetype=testSourceType basicQuery earliest=-2m latest=-1m
...

The difference between setting time in the dropdown and setting it in the query is the query overrides the dropdown.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...

Design, Compete, Win: Submit Your Best Splunk Dashboards for a .conf26 Pass

Hello Splunkers,  We’re excited to kick off a Splunk Dashboard contest! We know that dashboards are a primary ...

May 2026 Splunk Expert Sessions: Security & Observability

Level Up Your Operations: May 2026 Splunk Expert Sessions Whether you are refining your security posture or ...