Alerting

How do I make an alert for hosts that haven't sent their logs in?

splunktrainingu
Communicator

I am only curious for a certain index

index=abc
| stats count by host
| stats sum(count) AS Total BY host
| where Total>0

This search is good to see how many logs are coming in for my hosts in that index but the problem is when a host stops sending I have no alert for it. I tried changing the "|where Total>=0" but it took off the host from my table when it hit zero. How can I adjust or change my query to make it so I can alert when a host hits 0 logs.

Tags (1)
0 Karma

gjanders
SplunkTrust
SplunkTrust

If you instead are looking for a more global solution consider:
TrackMe
Or Meta Woot!
Or Broken hosts

I use TrackMe at the moment 🙂

0 Karma

splunktrainingu
Communicator

Thank you, my concern is does this app need to go on my forwarders or the is the app downloaded to app folder on my splunk indexer?

0 Karma

to4kawa
Ultra Champion
index=abc 
| stats count by host 
| append [| makeresults
| eval host=split("hostA,hostB,hostC ... ",",")
| mvexpand host
| fields host
| table host]
| stats sum(count) AS Total BY host | where Total>0

instead of append [....] , you can create host.csv and use | inputlookup append=t host.csv

0 Karma

splunktrainingu
Communicator

using this lookup table would it just have one column that being my hosts and the servers underneath it with no other columns?

0 Karma

to4kawa
Ultra Champion

host.csv

host
hostA
hostB
hostC
...

yes, it needs just one column host

0 Karma

splunktrainingu
Communicator

also where you stated

("hostA,hostB,hostC ... ",",")

do I place my hosts in there?

0 Karma

to4kawa
Ultra Champion
 | tstats count where index=abc by host
 | inputlookup append=t host.csv
 | stats sum(count) AS Total BY host | where Total>0
0 Karma

splunktrainingu
Communicator

Are the hosts in the csv case sensitive?

0 Karma

to4kawa
Ultra Champion

yes, case sensitive

| tstats count where index=_internal sourcetype="splunkd" by sourcetype 
| append [|makeresults
| eval sourcetype="SplunkD"]
| stats max(count) by sourcetype
0 Karma

splunktrainingu
Communicator

I am confused now, I don't need to eval the sourcetype I only care about my 3 hosts in my CSV that should be sending data and need it to alert when it doesn't.

0 Karma

to4kawa
Ultra Champion

my query on last comment is sample of case sensitive.
You should make the query yourself.

I won't make the query on your behalf.

0 Karma

splunktrainingu
Communicator

also I have more than 3 hosts for example I will have another csv file for my linux hosts and windows hosts and then for my high value hosts.

0 Karma

splunktrainingu
Communicator

I am going to make this lookup table then figure out how to change my query.

0 Karma
Get Updates on the Splunk Community!

New Case Study Shows the Value of Partnering with Splunk Academic Alliance

The University of Nevada, Las Vegas (UNLV) is another premier research institution helping to shape the next ...

How to Monitor Google Kubernetes Engine (GKE)

We’ve looked at how to integrate Kubernetes environments with Splunk Observability Cloud, but what about ...

Index This | How can you make 45 using only 4?

October 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...