Splunk Search

How to create an alert for events that are not logged?

Shashank_87
Explorer

Hi,

I have a weird requirement where I am looking to create an alert using some specific conditions. My OS index gets logged every 2 mins and I need to set up an alert to monitor a certain process "java" for a user "admin".

Now I have 4 hosts and suppose that 2 hosts (p3 and p4) are down, then trigger an alert using some specific message.

index=os sourcetype=ps host="p1" OR host="p2" OR host="p3" OR host="p4" user=admin process_name=java 
| stats count by host

Now under a normal scenario, the above search gives me 4 rows with a count of 1 for each host. In case my p3 and p4 goes down, I get only 2 rows because there is no process for p3 and p4.

Is there a simple way to achieve this?
I want to trigger an alert with a table that contains the following columns: host, message, priority.

The message would be common, something like, "These hosts are down. Please take action." and priority will also be always p2. But under the host column I need to specify those host which are down.

Labels (2)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

See https://www.duanewaddle.com/proving-a-negative/

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

See https://www.duanewaddle.com/proving-a-negative/

---
If this reply helps you, Karma would be appreciated.

Shashank_87
Explorer

Perfect. Cheers mate. That worked

0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...