Create alert when same warning keep coming for more than 5 hours

[JYTTEJ]

During maintenance we get the same application alert: MON001E on each trx. I want to set up an alert if application alert MON001E keep coming for more than 5 hours.

The search is to be scheduled to run each hour 24/7

If this application alert (MON001E) keep coming for more than 5 hours, SPLUNK must give an alert.

Example:no SPLUNK alert:

First MON001E at 01:00:00

Last MON001E at 05:00:00

Example: SPLUNK Alert:

First MON001E at 02:00

Last MON001E at 09:00

SPLUNK must alert at 08:00

Will this search do the trick?:

MID=MON001E earliest-6h latest=-1h

[joelshprentz · Accepted solution]

Try this search:

MID=MON001E earliest=-6h@h latest=@h | timechart span=1h count | eval count=min(1, count) | stats sum(count) as alertHours

Send an alert whenever alertHours > 5.

The search collects six hours of data matching MON001E. The timechart counts how many matches occur in each of the six hours. The eval limits those counts to be either 0 (no matches) or 1 (at least one MON001E). The stats totals the number of hours with at least one MON001E.

I interpreted your question as, "Send an alert whenever MID=MON001E occurs in six consecutive hours."

[JYTTEJ]

great! Thank you very much