I have a weird requirement where I am looking to create an alert using some specific conditions. My OS index gets logged every 2 mins and I need to set up an alert to monitor a certain process "java" for a user "admin".
Now I have 4 hosts and suppose that 2 hosts (p3 and p4) are down, then trigger an alert using some specific message.
index=os sourcetype=ps host="p1" OR host="p2" OR host="p3" OR host="p4" user=admin process_name=java
| stats count by host
Now under a normal scenario, the above search gives me 4 rows with a count of 1 for each host. In case my p3 and p4 goes down, I get only 2 rows because there is no process for p3 and p4.
Is there a simple way to achieve this? I want to trigger an alert with a table that contains the following columns: host, message, priority.
The message would be common, something like, "These hosts are down. Please take action." and priority will also be always p2. But under the host column I need to specify those host which are down.