How to create a dashboard for servers with a status Green/Red (Up or Down)

[Uday]

Hi, I got a new requirement to build a dashboard showing server status (Up/Down). Unfortunately our logs does n't indicate any such status like server started or server down. Any suggestions please? Any examples that I can see?

[asimit]

Hi @Uday,

There are several approaches to create a server status dashboard in Splunk when you don't have explicit "server up/down" logs. Here are the most effective methods:

## Method 1: Check for Recent Log Activity

This is the simplest approach - if a server is sending logs, it's probably up:

```
| metadata type=hosts index=*
| search host=*
| eval lastTime=strftime(recentTime,"%Y-%m-%d %H:%M:%S")
| eval status=if(now()-recentTime < 600, "UP", "DOWN")
| table host lastTime status
| sort host
```

Customize the time threshold (600 seconds = 10 minutes) based on your expected log frequency.

## Method 2: Using Rangemap for Visualization

Use rangemap to assign colors to status values:

```
| metadata type=hosts index=*
| search host=*
| eval lastTime=strftime(recentTime,"%Y-%m-%d %H:%M:%S")
| eval seconds_since_last_log=now()-recentTime
| eval status=if(seconds_since_last_log < 600, "UP", "DOWN")
| rangemap field=status up="0-0" down="1-1"
| table host lastTime status range
| sort host
```

For dashboard visualization, you'll need to add:

1. A CSS file (table_decorations.css) with content:
```css
.severe {
  background-color: #dc4e41 !important;
  color: white !important;
}

.low {
  background-color: #65a637 !important;
  color: white !important;
}
```

2. A JavaScript file (table_icons_rangemap.js) with content:
```javascript
require([
    'underscore',
    'jquery',
    'splunkjs/mvc',
    'splunkjs/mvc/tableview',
    'splunkjs/mvc/simplexml/ready!'
], function(_, $, mvc, TableView) {
    var CustomRangeRenderer = TableView.BaseCellRenderer.extend({
        canRender: function(cell) {
            return cell.field === 'range';
        },
        render: function($td, cell) {
            var value = cell.value;
            
            if(value === "severe") {
                $td.addClass('severe');
                $td.html('Down');
            } else if(value === "low") {
                $td.addClass('low');
                $td.html('Up');
            } 
            
            return $td;
        }
    });
    
    mvc.Components.get('table1').getVisualization(function(tableView) {
        tableView.addCellRenderer(new CustomRangeRenderer());
        tableView.render();
    });
});
```

3. Dashboard XML that includes these files:
```xml
<form script="table_icons_rangemap.js" stylesheet="table_decorations.css">
  <label>Server Status Dashboard</label>
  <fieldset submitButton="false">
    <input type="time" token="field1">
      <label></label>
      <default>
        <earliest>-60m@m</earliest>
        <latest>now</latest>
      </default>
    </input>
  </fieldset>
  <row>
    <panel>
      <table id="table1">
        <search>
          <query>| metadata type=hosts index=*
| search host=*
| eval lastTime=strftime(recentTime,"%Y-%m-%d %H:%M:%S")
| eval seconds_since_last_log=now()-recentTime
| eval status=if(seconds_since_last_log < 600, "UP", "DOWN")
| rangemap field=status up="0-0" down="1-1"
| table host lastTime status range
| sort host</query>
          <earliest>$field1.earliest$</earliest>
          <latest>$field1.latest$</latest>
        </search>
        <option name="drilldown">none</option>
      </table>
    </panel>
  </row>
</form>
```

## Method 3: Include All Expected Servers

To also show servers that aren't sending logs at all, use a lookup with all expected servers:

```
| inputlookup your_servers.csv
| append [| metadata type=hosts index=*]
| stats max(recentTime) as recentTime by host
| eval lastTime=if(isnotnull(recentTime),strftime(recentTime,"%Y-%m-%d %H:%M:%S"),"Never")
| eval seconds_since_last_log=if(isnotnull(recentTime),now()-recentTime,999999)
| eval status=if(seconds_since_last_log < 600, "UP", "DOWN")
| rangemap field=status up="0-0" down="1-1"
| table host lastTime status range
| sort host
```

## Method 4: Advanced Server Status Check (Recommended for Critical Systems)

If exact server status is critical, create a scheduled search that sends heartbeats from each server and alerts when they're missing:

1. Create a small script on each server that sends a heartbeat every few minutes:
```
index=server_status sourcetype=heartbeat host=$HOSTNAME$ status=ALIVE
```

2. Then use this search for your dashboard:
```
| inputlookup your_servers.csv
| map search="search earliest=-10m latest=now index=server_status sourcetype=heartbeat host=$host$ | head 1 | fields host"
| fillnull value="DOWN" status
| eval status=if(host=="NULL","DOWN","UP")
| rangemap field=status up="0-0" down="1-1"
| table host status range
```

This solution is more accurate than just checking for any logs, as it specifically monitors for heartbeat messages.

Remember to place your CSS and JS files in the /appserver/static/ directory of your app, and restart Splunk after adding them.

Please give 👍 for support 😁 happly splunking .... 😎

[gcusello]

Hi @Uday,

this is a new question and it's better to create a new one.

Anyway, you can see in Dashboard Examples (https://splunkbase.splunk.com/app/1603/) in the "Table Icon Set (Rangemap)" dashboard how to display status using an icon instead a value.

Is solves adding to your app a css and a js that are called by the dashboard

<form script="table_icons_rangemap.js" stylesheet="table_decorations.css">

then at your table is assigna an id:

 <table id="table1">

Remember to restart Splunk after you added css and js to the app and reload the page at every change in the dashboard otherwise you don't see the icons.

About the problem that you haven't a status field, you can create it using eval or (better) rangemap command, something like this:

e.g. to find hosts (listed in a lookup) that aren't sending logs:

| metasearch index=_internal
| eval host=lower(host)
| stats count BY host
| append [ | inputlookup systems.csv | eval host=lower(host), count=0 | fields host count ]
| stats sum(count) AS total BY host
| rangemap field=total severe=0-0 low=1-1000000000 default=severe

in this way the added javascript takes the value from rangemap and assign an icon.

Ciao.

Giuseppe

[Uday]

Thanks for your reply. Is it possible to provide me the query to search  logs for any data at all from the servers in the last x minutes? I will try this solution.

[richgalloway]

You could create a Python script that probes each server and sends an up/down indication to Splunk.

It may be easier, however, to search your logs for any data at all from the servers in the last x minutes.  If data is present then the server is up; otherwise, consider it down.

[vectordba]

create a python script to collect the logs and send it to splunk?
why even use splunk then?
if you have log collection method and a bootstrap dashboard with a small local database; you have a full fledged monitoring app.  there are a thousand monitoring apps out there.  whats one more?

[isoutamo]

There are several tools to do server monitoring stuff. To be honest core splunk is not one of the best tools for that task. Core Splunk's best part is to manage those logs and also integration to other Splunk's tools/servers which are much better for that tasks.

From Splunk offering you should look this https://www.splunk.com/en_us/products/infrastructure-monitoring.html for pure server monitoring.

[Uday]

Thanks for your reply. Is it possible to provide me the query to search  logs for any data at all from the servers in the last x minutes? I will try this solution.