Splunk Search

i need and when a splunk agent goes down or stops

rndp89
Explorer

i have 30 servers, out of which I want to monitor splunk agents of only 4 servers

i have the following query.

index=_internal data_host=server1 OR data_host=server2 OR data_host=server3 OR data_host=server4 | stats count by data_host

how to create alert for the server which doesn't show up in the result table?
Alert should contain the server name which is not/stopped reporting data to splunk controller

For example:- server2 went down or the splunk agent of server2 went down for some reason.
Alert should say "Server2 is not reporting data"

Tags (2)
0 Karma

niketn
Legend

@rndp89 by Splunk Agent do you mean when Universal Forwarders on Deployment Clients stop working?

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

mayurr98
Super Champion

hey try this

| metadata type=hosts index=* OR index=_internal 
| eval age = now() - recentTime 
| eval status= case(age < 1800,"Running",age > 1800,"DOWN") 
| convert ctime(recentTime) AS LastActiveOn 
| eval age=tostring(age,"duration") 
| eval host = upper(host) 
| table host age LastActiveOn status 
| rename host as "Forwarder Name", age as "Last Heartbeat(min)",LastActiveOn as "Last Active On",status as Status 
| where Status= "DOWN"

OR you can have a look at this answer:
https://answers.splunk.com/answers/318037/how-to-create-an-alert-to-trigger-an-email-when-a.html

let me know if this helps!

Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...