Alert Set up

[smuderasi]

I want to set up an alert  when the server status is not "HEALTH_OK" for three consecutive times in a row.

Any pointers on how to schedule such search. I receive data every two minutes.

Send Alert If	Comments/Alert Frequency
Status is not "RUNNING" or State is not "HEALTH_OK" for 3 times consecutively	1 alert every 2 min, for first 10 min, then 1 alert every 30 min

[rnowitzki]

Hi @smuderasi ,

This is not the finished meal, but should give you an idea of how you can do it:

| noop 
| makeresults 
| eval id = "1 2 3 4 5 6 7 8 9" 
| makemv id
| mvexpand id
| eval server = case(id=1,"hostA",id=2,"hostB",id=3,"hostC",id=4,"hostA",id=5,"hostC",id=6,"hostB",id=7,"hostB",id=8,"hostA",id=9,"hostC")
| eval _time = case(id=1,_time+10,id=2,_time+20,id=3,_time+40,id=4,_time+40,id=5,_time+50,id=6,_time+60,id=7,_time+70,id=8,_time+80,id=9,_time+90)
| eval state = case(id=1,"health_ok",id=2,"health_not_ok",id=3,"health_ok",id=4,"health_ok",id=5,"health_ok",id=6,"health_not_ok",id=7,"health_not_ok",id=8,"health_not_ok",id=9,"health_not_ok")
| fields - id

| sort server, _time
| streamstats count as alert_counter by server, state

The first block, until including the | fields - id is just to make up some sample data - you don't need that as you have data 🙂

The streamstats will create the field alert_counter which adds 1 every time the server and state is the same as in the line above. So your alert could trigger when alert_counter>=3 AND state!=health_ok
You can also add your status field like that.

I don't have a good idea right now for your alert conditions (every 2 minutes, later every 30 minutes), might be possible with 2 different alerts somehow.

Hope I could at least give you some inspiration.