Splunk Search

How to crete an alert to notify when host back to normal?

smanojkumar
Contributor

I'm having a list of serve down and need to notify once its back to normal (up),  This is the requirement,

once the server is up, no need to consider the same server further, because its already up , need to check the remaining.

Eg.., There are servers A,B,C,D and E are down, which will be there in lookup,

Need to check those server every minute and notify once its up,

if server A,B is up after some time,  then it should trigger an alert, already server A,B is up, and after next alert, server A,B should not be considered, only remaining servers like C,D and E should be considered further, Then it check and trigger alert when C, D and E or either one is up.


index=linux sourcetype=df
| lookup Hobbit_threshold_data host mount outputnew l_threshold as lower_value h_threshold as higher_value condition as Condition
| where ((PercentUsedSpace >= lower_value) AND (PercentUsedSpace<higher_value))
| where Condition!="no"
| eval hostname=mvindex(split(host,"."),0) [ | inputlookup Hobbit_Disk_Space_Warning.csv | fields host ]
| stats host=lower(host)
| stats count BY host
| append [ | inputlookup KCI_Hobbit_Disk_Space_Warning.csv | eval host=lower(host), count=0 | fields host count ]
| stats sum(count) AS total BY host
| eval status=if(total=0,"Down","Up")

(Bolded query part gives you server down list host)
I just modified as per the query given by you,  It does not meet the requirement.

Labels (1)
0 Karma
Get Updates on the Splunk Community!

Index This | Why did the turkey cross the road?

November 2025 Edition  Hayyy Splunk Education Enthusiasts and the Eternally Curious!   We’re back with this ...

Enter the Agentic Era with Splunk AI Assistant for SPL 1.4

  &#x1f680; Your data just got a serious AI upgrade — are you ready? Say hello to the Agentic Era with the ...

Feel the Splunk Love: Real Stories from Real Customers

Hello Splunk Community,    What’s the best part of hearing how our customers use Splunk? Easy: the positive ...