I'd like to be able to check on the status of the splunkweb process from distributed splunk instances within a splunk search. I could easily do this with a script but I want to avoid at all costs. I need an internal splunk method to obtain the status of remote splunk installations.
At the moment I can simply query splunkd processes using a rest command such as "| rest /services/server/info" and I can be sure that the splunkd process on all the listed hosts is working as they are returning info about themselves.
I have had situations in the past where splunkweb or splunkd would independently crash so i'd like to make a dashboard + alerts that would inform us of this (without use of scripts!!!!)
You can get the process IDs of the running splunkweb processes in your distributed environment using the following:
| rest /services/server/status/resource-usage/splunk-processes | search process=splunkweb | table splunk_server pid
You can adapt this to get a count and alert off that...
Thanks for the reply i'd been waiting aaages to try and find something like this but it doesn't seem to work on my installs. What version of splunk was that run on?
edit: If I walk the path I can see
| rest /services/server/status/resource-usage
Inside there I get hostwide and splunk-processes but they both don't contain anything.