Alert when both devices in HA pair stop reporting

[gstefancyk]

I currently have an alert setup to send email when one of our infoblox servers stops sending logs based on metadata of the index and a csv lookup for hosts. This works well but the issue I am running into is that some of the devices are in an HA pair so I am constantly getting emails for the backup device not reporting. Below is my current search,

| metadata type=hosts index=infoblox
| lookup infoblox_hosts.csv host_ip as host OUTPUT host_ip as host, server_name as server
| search host=*
| where lastTime < (now() - 86400)
| convert ctime(lastTime) as LastTimeLogged
| table host,server LastTimeLogged
| sort LastTimeLogged

Can someone point me in the right direction to setup a condition to alert when both server1 and server2 have stopped reporting otherwise do nothing?

Thanks

[maciep · Accepted solution]

is there anything in your lookup that identifies which servers are part of the HA pair? maybe another column that identifies the HA IP for example?

If so, then you might be able to just worry about the latest event from that vip. Something like:

| metadata type=hosts index=infoblox 
| lookup infoblox_hosts.csv host_ip as host OUTPUT host_ip as host, server_name as server, ha_ip as vip
| search host=*
| eventstats max(lastTime) as vip_lastTime by vip
| where vip_lastTime < (now() - 86400)
| convert ctime(vip_lastTime) as LastTimeLogged
| table host,server LastTimeLogged

[gstefancyk]

Thanks for the suggestion, I will give this a try and see happens.

[gstefancyk]

Thanks maciep, works as expected after adding 3rd column to my lookup table.