Getting Data In

How to create a live up or down dashboard view?

Path Finder

sphiwee_0-1594045928296.png

Good day is it possible to get above visualization on Splunk? im kinda new and lost, I designed this myself using PowerPoint. We want to get a view of all the services running in our servers to be like the above image on Splunk dashboards.  We are currently running a cron script on our server and forwarding the results to our Splunk server every 5 minutes.

sphiwee_1-1594046342208.png

The results look like the above screenshot. We want a live view of all the services running with green being up and red being down.

Labels (3)
Tags (2)
0 Karma
1 Solution

Motivator

Ok, I would first start with verifying that either the "RUNNING"/"NOT RUNNING"  values (I'm assuming there is an inverse value to RUNNING) are recognized by Splunk as a field. You can check apps.splunk.com to see if there is an addon that will parse your sourcetype into key/value pairs or you may have to write regex to capture the value of the run state into a field. Once it's in a field you can run statistic commands against that field such as | top state by host

example inline regex command to extract the state

| rex "(?<state>(RUNNING|DOWN))"

 

extract-fields.jpg

If you are getting overwhelmed, you may want to start with one of the free classes which will cover fields in Splunk https://www.splunk.com/en_us/training/free-courses/splunk-fundamentals-1.html

I also noticed you are running a script from crontab. You could modify the format of your log to log in key-pair values. i.e. process_status="RUNNING" or process_status="UP" or process_status="DOWN" etc.

When Splunk encounters key/pair values it will auto extract fields which should make this task much simpler.

https://dev.splunk.com/enterprise/docs/developapps/logging/loggingbestpractices/

View solution in original post

Motivator

your base query | dedup host application | chart values(state) over host by application

 

note: modify fields names to match your dataset

 

then you can color code in a table or single value chart using trellis option

color-code-table-values.jpg

 

 

0 Karma

Path Finder

@rob_jordan1 thank you sir, what do I add to my base query? all the services? separated by OR ?

0 Karma

Motivator

I'm not familiar with your specific dataset, however you would want to come up with a base query that matches the events in Splunk that have the states you are trying to track.

 

i.e.

index=myindex sourcetype=mysourcetype host IN (host1,host2,host3) service IN (service1,service2,service3)

note: you can wildcard parts of the host or service filters with * as well. If wanting all hosts and services, you probably don't need to add a host or service constraint.

Path Finder

@rob_jordan1 typing your solution

sphiwee_1-1594052932087.png

 

heres what my data looks like, circled in black is what i could use to get the up status, sorry for this but the documentation is also not helping

sphiwee_2-1594053267181.png

 

 

0 Karma

Motivator

Ok, I would first start with verifying that either the "RUNNING"/"NOT RUNNING"  values (I'm assuming there is an inverse value to RUNNING) are recognized by Splunk as a field. You can check apps.splunk.com to see if there is an addon that will parse your sourcetype into key/value pairs or you may have to write regex to capture the value of the run state into a field. Once it's in a field you can run statistic commands against that field such as | top state by host

example inline regex command to extract the state

| rex "(?<state>(RUNNING|DOWN))"

 

extract-fields.jpg

If you are getting overwhelmed, you may want to start with one of the free classes which will cover fields in Splunk https://www.splunk.com/en_us/training/free-courses/splunk-fundamentals-1.html

I also noticed you are running a script from crontab. You could modify the format of your log to log in key-pair values. i.e. process_status="RUNNING" or process_status="UP" or process_status="DOWN" etc.

When Splunk encounters key/pair values it will auto extract fields which should make this task much simpler.

https://dev.splunk.com/enterprise/docs/developapps/logging/loggingbestpractices/

View solution in original post