How to create search that pulls the server status of running vs. shutdown?

[Uday]

Can you please help me with a search to display a list of servers with a status Running or Shutdown?

I have a list of hostnames, but I am not sure how to show if the server status is Running or Shutdown. Eventually I have to build a dashboard out of it.

[gcusello]

Hi @Uday,

I can image that you have the Splunk UNiversal Forwarder installed in each of the monitored servers, so you can monitor the internal Splunk logs to know if the server is up or down.

You have to put the servers to monitor list in a lookup (called e.g. perimeter) with at least one field contaning hostnames (called host).

With these premises, you can run something like this:

| metasearch index=_internal
| eval host=lower(host)
| stats count BY host
| append [ | inputlookup perimeter | eval host=lower(host), count=0 | fields host count ]
| stats sum(count) AS total BY host
| eval status=if(total=0,"Down","Up")
| table host status

You can display the results in a dashboard, that you can also have in graphic mode following the example "Table Icon Set (Rangemap)" that you can find in the Splunk Dashboard Examples app (https://splunkbase.splunk.com/app/1603/).

Or simply in an alert deleting the last two rows and adding the condition | where total=0

| metasearch index=_internal
| eval host=lower(host)
| stats count BY host
| append [ | inputlookup perimeter | eval host=lower(host), count=0 | fields host count ]
| stats sum(count) AS total BY host
| where total=0

Ciao.

Giuseppe

[Uday]

Can you please help me to plug in the rangemap query to the below ?

| metasearch index=_internal
| eval host=lower(host)
| stats count BY host
| append [ | inputlookup perimeter | eval host=lower(host), count=0 | fields host count ]
| stats sum(count) AS total BY host
| eval status=if(total=0,"Down","Up")
| table host status

[gcusello]

Hi @Uday,

please, try this:

| metasearch index=_internal
| eval host=lower(host)
| stats count BY host
| append [ | inputlookup perimeter | eval host=lower(host), count=0 | fields host count ]
| stats sum(count) AS total BY host
| rangemap field=total severe=0-0 low=1-1000000000 default=severe
| eval status=if(total=0,"Down","Up")
| table host status range

In the table you can eventually choose to display only host and range.

You can also display results in graphic mode, following the instruction in"Table Icon Set (Rangemap)" dashboard in the "Splunk Dashboard Examples" App (https://splunkbase.splunk.com/app/1603/).

Ciao.

Giuseppe

[Uday]

I just tried this and the query helped. Can you please help with the query using range map to display the status Up/Down on the dashboard?

[Uday]

Thank you. Is there a way we can do it based on the last event in the log?

[gcusello · Accepted solution]

Hi @Uday,

as I said, you use the Splunk internal logs that are always present.

Using | metasearch you have a very quick search, so you don't need to use only one event.

You can run this search as an alert every e.g. 5 minutes on a timeframe of 5 minutes having a very quick answer.

In this way you almost immediately have an alert when a server is down.

Ciao.

Giuseppe

[richgalloway]

Do you have events in Splunk that show the status of the servers?

[Uday]

No, unfortunately the logs does n't have any thing that indicates what time the server started or shutdown.