Splunk Search

How to create an alert for events that are not logged?

Shashank_87
Explorer

Hi,

I have a weird requirement where I am looking to create an alert using some specific conditions. My OS index gets logged every 2 mins and I need to set up an alert to monitor a certain process "java" for a user "admin".

Now I have 4 hosts and suppose that 2 hosts (p3 and p4) are down, then trigger an alert using some specific message.

index=os sourcetype=ps host="p1" OR host="p2" OR host="p3" OR host="p4" user=admin process_name=java 
| stats count by host

Now under a normal scenario, the above search gives me 4 rows with a count of 1 for each host. In case my p3 and p4 goes down, I get only 2 rows because there is no process for p3 and p4.

Is there a simple way to achieve this?
I want to trigger an alert with a table that contains the following columns: host, message, priority.

The message would be common, something like, "These hosts are down. Please take action." and priority will also be always p2. But under the host column I need to specify those host which are down.

Labels (2)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

See https://www.duanewaddle.com/proving-a-negative/

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

See https://www.duanewaddle.com/proving-a-negative/

---
If this reply helps you, Karma would be appreciated.

Shashank_87
Explorer

Perfect. Cheers mate. That worked

0 Karma
Get Updates on the Splunk Community!

Splunk Observability for AI

Don’t miss out on an exciting Tech Talk on Splunk Observability for AI!Discover how Splunk’s agentic AI ...

Splunk Enterprise Security 8.x: The Essential Upgrade for Threat Detection, ...

Watch On Demand the Tech Talk, and empower your SOC to reach new heights! Duration: 1 hour  Prepare to ...

Splunk Observability as Code: From Zero to Dashboard

For the details on what Self-Service Observability and Observability as Code is, we have some awesome content ...