Getting Data In

How to create a live up or down dashboard view?

sphiwee
Contributor

sphiwee_0-1594045928296.png

Good day is it possible to get above visualization on Splunk? im kinda new and lost, I designed this myself using PowerPoint. We want to get a view of all the services running in our servers to be like the above image on Splunk dashboards.  We are currently running a cron script on our server and forwarding the results to our Splunk server every 5 minutes.

sphiwee_1-1594046342208.png

The results look like the above screenshot. We want a live view of all the services running with green being up and red being down.

Labels (3)
Tags (2)
0 Karma
1 Solution

bandit
Motivator

Ok, I would first start with verifying that either the "RUNNING"/"NOT RUNNING"  values (I'm assuming there is an inverse value to RUNNING) are recognized by Splunk as a field. You can check apps.splunk.com to see if there is an addon that will parse your sourcetype into key/value pairs or you may have to write regex to capture the value of the run state into a field. Once it's in a field you can run statistic commands against that field such as | top state by host

example inline regex command to extract the state

| rex "(?<state>(RUNNING|DOWN))"

 

extract-fields.jpg

If you are getting overwhelmed, you may want to start with one of the free classes which will cover fields in Splunk https://www.splunk.com/en_us/training/free-courses/splunk-fundamentals-1.html

I also noticed you are running a script from crontab. You could modify the format of your log to log in key-pair values. i.e. process_status="RUNNING" or process_status="UP" or process_status="DOWN" etc.

When Splunk encounters key/pair values it will auto extract fields which should make this task much simpler.

https://dev.splunk.com/enterprise/docs/developapps/logging/loggingbestpractices/

View solution in original post

bandit
Motivator

your base query | dedup host application | chart values(state) over host by application

 

note: modify fields names to match your dataset

 

then you can color code in a table or single value chart using trellis option

color-code-table-values.jpg

 

 

0 Karma

sphiwee
Contributor

@bandit thank you sir, what do I add to my base query? all the services? separated by OR ?

0 Karma

bandit
Motivator

I'm not familiar with your specific dataset, however you would want to come up with a base query that matches the events in Splunk that have the states you are trying to track.

 

i.e.

index=myindex sourcetype=mysourcetype host IN (host1,host2,host3) service IN (service1,service2,service3)

note: you can wildcard parts of the host or service filters with * as well. If wanting all hosts and services, you probably don't need to add a host or service constraint.

sphiwee
Contributor

@bandit typing your solution

sphiwee_1-1594052932087.png

 

heres what my data looks like, circled in black is what i could use to get the up status, sorry for this but the documentation is also not helping

sphiwee_2-1594053267181.png

 

 

0 Karma

bandit
Motivator

Ok, I would first start with verifying that either the "RUNNING"/"NOT RUNNING"  values (I'm assuming there is an inverse value to RUNNING) are recognized by Splunk as a field. You can check apps.splunk.com to see if there is an addon that will parse your sourcetype into key/value pairs or you may have to write regex to capture the value of the run state into a field. Once it's in a field you can run statistic commands against that field such as | top state by host

example inline regex command to extract the state

| rex "(?<state>(RUNNING|DOWN))"

 

extract-fields.jpg

If you are getting overwhelmed, you may want to start with one of the free classes which will cover fields in Splunk https://www.splunk.com/en_us/training/free-courses/splunk-fundamentals-1.html

I also noticed you are running a script from crontab. You could modify the format of your log to log in key-pair values. i.e. process_status="RUNNING" or process_status="UP" or process_status="DOWN" etc.

When Splunk encounters key/pair values it will auto extract fields which should make this task much simpler.

https://dev.splunk.com/enterprise/docs/developapps/logging/loggingbestpractices/

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...