topic Re: Splunk Query to pull the server status in Splunk Search

How to create search that pulls the server status of running vs. shutdown?

Uday — Wed, 02 Sep 2020 23:11:42 GMT

Can you please help me with a search to display a list of servers with a status Running or Shutdown?

I have a list of hostnames, but I am not sure how to show if the server status is Running or Shutdown. Eventually I have to build a dashboard out of it.

Re: Splunk Query to pull the server status

richgalloway — Wed, 02 Sep 2020 15:05:41 GMT

Do you have events in Splunk that show the status of the servers?

Re: Splunk Query to pull the server status

Uday — Wed, 02 Sep 2020 15:11:26 GMT

No, unfortunately the logs does n't have any thing that indicates what time the server started or shutdown.

Re: Splunk Query to pull the server status

gcusello — Wed, 02 Sep 2020 15:13:02 GMT

Hi @Uday,

I can image that you have the Splunk UNiversal Forwarder installed in each of the monitored servers, so you can monitor the internal Splunk logs to know if the server is up or down.

You have to put the servers to monitor list in a lookup (called e.g. perimeter) with at least one field contaning hostnames (called host).

With these premises, you can run something like this:

You can display the results in a dashboard, that you can also have in graphic mode following the example "Table Icon Set (Rangemap)" that you can find in the Splunk Dashboard Examples app (https://splunkbase.splunk.com/app/1603/).

Or simply in an alert deleting the last two rows and adding the condition | where total=0

Ciao.

Giuseppe

Re: Splunk Query to pull the server status

Uday — Wed, 02 Sep 2020 21:01:48 GMT

Thank you. Is there a way we can do it based on the last event in the log?

Re: Splunk Query to pull the server status

gcusello — Thu, 03 Sep 2020 07:11:41 GMT

Hi @Uday,

as I said, you use the Splunk internal logs that are always present.

Using | metasearch you have a very quick search, so you don't need to use only one event.

You can run this search as an alert every e.g. 5 minutes on a timeframe of 5 minutes having a very quick answer.

In this way you almost immediately have an alert when a server is down.

Ciao.

Giuseppe

Re: Splunk Query to pull the server status

Uday — Thu, 03 Sep 2020 17:20:10 GMT

I just tried this and the query helped. Can you please help with the query using range map to display the status Up/Down on the dashboard?

Re: Splunk Query to pull the server status

Uday — Thu, 03 Sep 2020 17:28:15 GMT

Can you please help me to plug in the rangemap query to the below ?

| metasearch index=_internal
| eval host=lower(host)
| stats count BY host
| append [ | inputlookup perimeter | eval host=lower(host), count=0 | fields host count ]
| stats sum(count) AS total BY host
| eval status=if(total=0,"Down","Up")
| table host status

Re: Splunk Query to pull the server status

gcusello — Fri, 04 Sep 2020 06:57:52 GMT

Hi @Uday,

please, try this:

In the table you can eventually choose to display only host and range.

You can also display results in graphic mode, following the instruction in"Table Icon Set (Rangemap)" dashboard in the "Splunk Dashboard Examples" App (https://splunkbase.splunk.com/app/1603/).

Ciao.

Giuseppe