Current Alert Setup:
I am trying to set up an alert to send an email when the response time from the server is higher (>60ms). I have the webpage running on 4 hosts.
Search string:
index=iserver env=prod sourcetype="iis-access" uri_path="index.html" code=200 | where time_taken > 60
Alert Type: Real-time.
Trigger Condition: Number of Results is > 1 in 5 minutes. Edit
When triggered, execute actions: For each result.
I have a throttle setup for the field 'host' for 2 minutes. I do not want the same host to be reported for next 2 minutes at least.
Problem: The alert triggers perfectly and shoots an email only once for each result after setup and for the rest of the day, I do not get any email alerts. But the search returns results when I open it in search in real-time.
Can someone help me identify where am I getting it wrong?
Thanks,
Check the EXPIRATION time of your alert.It may have been expired.
I hope you are referring editing below parameter in $SPLUNK_BASE/etc/system/local/savedsearches.conf file.
alert.expires = <new_value>
# it was 24h in the defaults