I want to set up splunk alert that can have two threshold
1. if the time is between 8 AM to 5PM - alert if AvgDuration is greater than 1000ms
2. If time is between 5pm to next day 8AM - alert if avgduration is greater than 500ms
How do i implement this
Query am working on
<mySearch>| bin _time span=1m|
stats avg(msg.DurationMs) AS AvgDuration by _time, msg.Service | where AvgDuration > 1000
You could calculate the current hour of the alert execution, then adjust the threshold at the end.
<mySearch>
| bin _time span=1m
| stats avg(msg.DurationMs) AS AvgDuration by _time, msg.Service
| eval hour = strftime(now(),"%H")
| where (AvgDuration > 1000 and hour >= 8 and hour < 17) or (AvgDuration > 500 AND (hour < 8 OR hour >= 17))