How to monitor a windows service, send an alert and restart the service?

[mtoddsmith]

How can we to monitor various windows services and send alerts when they are down and optionally attempt to restart the service via splunk.

[dwaddle]

We accomplish half of this via WMI. Add this to wmi.conf and push it out via deployment server (or however you push out configs)

[WMI:Services]
interval = 60
disabled = 0
index = default
wql = select Name, ProcessId, Caption, DisplayName, State, Status, StartName, SystemName from Win32_Service

Once you're collecting data, then searching/alerting on services that are not in the right status is pretty simple.

[noy72]

I have added the additional displaynames to my WMI.conf file on the SCCM forwarder, I have veified that the WMI namespace on SCCM has theapropriate permissions and I have verified that Splunk is receiving WMI data. Can someone help me out with a query to veify the status of SCCM specific services?
Thank you
Ron Jones

[noy72]

After verifing the above, al searches started working. I apreciate the assistance.
Ron Jones

[Chubbybunny · Accepted solution]

use the sc. exe command to check on the status of each service.

DESCRIPTION:
SC is a command line program used for communicating with the
NT Service Controller and services.

C:\Documents and Settings\chubbybunny.hare>sc.exe query SplunkForwarder

SERVICE_NAME: SplunkForwarder
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 4  RUNNING
                                (STOPPABLE,NOT_PAUSABLE,ACCEPTS_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0

craft a splunk search for 'STATE STOPPED'

C:\Documents and Settings\chubbybunny.hare>sc.exe query SplunkForwarder

SERVICE_NAME: SplunkForwarder
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 1  STOPPED
                                (NOT_STOPPABLE,NOT_PAUSABLE,IGNORES_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0

add an Alert action to 'Run a script' to start the service

C:\Documents and Settings\chubbybunny.hare>sc.exe start SplunkForwarder

SERVICE_NAME: SplunkForwarder
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 2  START_PENDING
                                (NOT_STOPPABLE,NOT_PAUSABLE,IGNORES_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x7d0
        PID                : 4000
        FLAGS              :

upVote the Chubbybunny if it helps!

(\__/)
(='.'=)
(")_(")