How do you send an alert only when a condition is met consecutively?

[thezen]

Hi,

I have the following alert set up: send an alert if there are more than 5 matching events for every server within the last 5 minutes.

Now, I need to update the alert such that it will be sent out only if the condition matches twice in a row; note that I can't just put the condition to alert when there are 10 events within the last 10m, since the idea is to only alert if the number of matching event is consistently above 5 every 5 minutes.

This is the closest query I can get, but I still can't get rid of the server, which doesn't satisfy the condition..

index=myindex source=logA | timechart usenull=f useother=f limit=0 span=5m count by SERVER

The above will neatly give stats for every 5 minute window for every server, so now I just need to somehow filter out the server that doesn't match the condition, so then I could create the alert to run every 10m to trigger when there's more than 1 event.

Appreciate any suggestions,

[macadminrohit]

Can you show the trigger search that you already have, i do the similar stuff in our environment. The logic is to mark each event after timechart as greater than 5 or not . and then finally calculating the number of events where you have yes. Below is the sample search, let us know if it works for you. I can help further.

| makeresults count=10 
| eval count=1 
| timechart span=5m count
| eval server_count_gt_1=if(count>5,"Yes","No") 
| timechart span=10m count(eval(server_count_gt_1="Yes")) as count_more_than2

[richgalloway]

How about index=myindex source=logA | timechart usenull=f useother=f limit=0 span=5m count by SERVER | where count > 4?