Alerting

If we have an alert that tells us when a service is stopped, how can we create an alert that will also check if the service remains down?

luise
New Member

Hello,

We have an alert that tells us when a service stopped. However, the services will sometimes restart. How can we have an alert that will also check if the service remains down?

Is it possible?

Thanks,
Luis Embalo

0 Karma

sundareshr
Legend

Maybe something like this might work

index=* "success" earliest=[search index=* "failed" | timechart latest(failed) as earliest | fields earliest] | stats count 

If the count is 0 its probably because the service hasn't restarted.

0 Karma

luise
New Member

That searches for a service that is down.. what I am looking for is for way that sees the service as going down, and then keeps watching to see if it came up. If it came up, then nothing happens.. if it did not come up, then alerts.

Is that possible?

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

Tech Talk Recap | Mastering Threat Hunting

Mastering Threat HuntingDive into the world of threat hunting, exploring the key differences between ...

Observability for AI Applications: Troubleshooting Latency

If you’re working with proprietary company data, you’re probably going to have a locally hosted LLM or many ...

Splunk AI Assistant for SPL vs. ChatGPT: Which One is Better?

In the age of AI, every tool promises to make our lives easier. From summarizing content to writing code, ...