Deployment Architecture

How to find stopped service ?

ethammis
Engager
  • I have windows service called "ess". Due to network glitch the service is entering into stopped state and start state. Since the windows event is generating for delivery network glitch an event is recorded in splunk.

But the service ess is really down, and never entered into running state we need to be alerted.

I want to write splunk to alert only when the service ess went into stopped state but never entered into running state for 25 hosts. Same service is running on 25 hosts and all servers has network glitches.

Labels (2)
0 Karma
1 Solution

venkateshparank
Path Finder

Try something below:

index=<indexname> ("ess service")
|transaction host startswith="The ess service entered the stopped state." endswith="The ess service entered the running state." maxspan=30m
|search NOT <field>="The ess service entered the running state."
|table host

View solution in original post

venkateshparank
Path Finder

Try something below:

index=<indexname> ("ess service")
|transaction host startswith="The ess service entered the stopped state." endswith="The ess service entered the running state." maxspan=30m
|search NOT <field>="The ess service entered the running state."
|table host

ethammis
Engager

How long are you prepared to wait for the service to come up again?

- Within 10min, if service is not coming up then need alert.

i.e. an event "The ess service entered into running state" will be logged

Are you looking to alert if all the servers don't come back up within a certain time, or if any one of them doesn't come back up?
Any server out of 25, if the service is not running, then need alert

Are events generated when the service is up, and how regularly do these events occur?

As soon the service started an event will be generated, "The ess Service entered into running state"

Can there be periods when no events are generated but the service is still to be considered up?

No, there will be definitely an event will be generated once the service brought up

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

How long are you prepared to wait for the service to come up again? Are you looking to alert if all the servers don't come back up within a certain time, or if any one of them doesn't come back up? Are events generated when the service is up, and how regularly do these events occur? Can there be periods when no events are generated but the service is still to be considered up?

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...

Design, Compete, Win: Submit Your Best Splunk Dashboards for a .conf26 Pass

Hello Splunkers,  We’re excited to kick off a Splunk Dashboard contest! We know that dashboards are a primary ...

May 2026 Splunk Expert Sessions: Security & Observability

Level Up Your Operations: May 2026 Splunk Expert Sessions Whether you are refining your security posture or ...