Splunk IT Service Intelligence
Highlighted

How to add to ITSI dashboard the Status of services/ processes running in the OS?

Path Finder

One requirement is monitoring the Status of Services/processes running in operating systems. Using Splunk ITSI is there a way to do provide the necessary data input to display as an ITSI monitored icon?

Tags (2)
0 Karma
Highlighted

Re: How to add to ITSI dashboard the Status of services/ processes running in the OS?

Path Finder

The key point here is that ITSI works entirely off numeric values and with this in mind if your script or routine returns a numeric value it can be used in ITSI's dashboard.

So a quick google search turns up the utility sc on windows to query services and their running status

sc query

without arguments it returns a list of services and details about them including their current state. The state contains a numeric value that you could extract and use in ITSI. In this example a running service is showing 4 and a stopped service is showing 1. You can then assign a threshold for your KPI where above 3 is started / green and below 2 is failed and stopped. Anything in between could be orange where the service is either starting up or stopping.

SERVICENAME: wuauserv
DISPLAY
NAME: Windows Update
TYPE : 20 WIN32SHAREPROCESS
STATE : 4 RUNNING
(STOPPABLE, NOTPAUSABLE, ACCEPTSPRESHUTDOWN)
WIN32EXITCODE : 0 (0x0)
SERVICEEXITCODE : 0 (0x0)
CHECKPOINT : 0x0
WAIT_HINT : 0x0

An example of a stopped process.

C:\Users\Administrator>sc query ALG

SERVICENAME: ALG
TYPE : 10 WIN32
OWNPROCESS
STATE : 1 STOPPED
WIN32
EXITCODE : 1077 (0x435)
SERVICE
EXITCODE : 0 (0x0)
CHECKPOINT : 0x0
WAIT
HINT : 0x0

For processes in windows you have "tasklist" so if the process is present in the list it has a 1 and if not a 0.

For linux you can also use the process table to check if it is running [ps -eaf] and most services in linux have a status command so although painful you could run it for each service you need to check. Again you need to select for a numeric criteria and based on this criteria generate a number that can be passed to ITSI.

e.g. $SPLUNK_HOME/bin/splunk status

0 Karma