Is there a way to apply condition to Windows Monitoring Dashboard?

[annbingle]

Hi and Good day!

I wrote a PSI script to filter Windows Services I need to monitor in our Splunk OBS dashboard. However,  Splunk detects each Windows service as one value. For example,  the script gathers 10 windows services. 5 of them are Running and 5 has stopped.

In my dashboard I want to show all the Windows services. If the service is running then it should be green and if it stopped then it should be red. Right now, everything is showing the same Color, the same value for both running  and stopped.

Is there a way to apply this condition in the dashboard?

Thank you in advance and looking forward to your response.

Have a great day!
Ann

[annbingle]

Hi,

Here's the sample logs I generate using the the script I made. I just want all these services displayed in the dashboard. Services that are running should be green and the Services that are STOPPED should be red. I read somewhere that we can apply conditional logic in the Splunk dashboard but I am not sure how to do it. 

Thank you so much for taking your time in trying to help me 🙂

Service_A Running INFO
Service_B Running INFO
Service_C STOPPED ERROR
Service_D STOPPED ERROR
Service_E Running INFO

[gcusello]

Hi @annbingle,

I suppose that each row is a different event with its own timestamp and that all the services are for a single host.

In this case you should list all services in the lookup as described above and you could could run something like this:

index=your_index
| rex "^(?<service>\w*)\s+(?<status>\w*)"
| eval service=lower(service)
| stats count BY service
| append [ | inputlookup services.csv | | eval service=lower(service), count=0 | fields service count ]
| stats sum(count) AS total values(status) AS status BY service
| eval status=if(total>0 OR status="Running","Running","STOPPED")
| table service status

Tghis solution is required to check if there's some service that is under monitoring but isn't listed in your events.

If instead you want only to group the stopped services from your logs, you have only to run:

index=your_index
| rex "^(?<service>\w*)\s+(?<status>\w*)"
| search status="STOPPED"
| table service status

Ciao.

Giuseppe

[annbingle]

Thank you @gcusello !Let me test on this then. Thank you! Appreciate it!

Have a great day!

Ann

[gcusello]

Hi @annbingle,

what is the issue in your search: maybe in your data there are only the running services and you want also the not running ones?

if this is yur requirement, you have only to create a lookup (called e.g. services.csv) containing all the services to monitor (at least two columns: host, service) and run a search like the following:

index=your_index
| eval host=lower(host), service=lower(service)
| stats count BY host service
| append [ | inputlookup services.csv | | eval host=lower(host), service=lower(service), count=0 | fields host service count ]
| stats sum(count) AS total BY host service
| eval status=if(total="Not running","Running")
| table host service status

then you can associate a colour or an icon to the status following the instructions in the Splunk Dashboard Examples App (https://splunkbase.splunk.com/app/1603) Table Icon Set (Rangemap).

I don't know the events, if you could share your events I could be more detailed in the search.

Ciao.

Giuseppe

[annbingle]

Hi Giuseppe,

Thank you for your response. My data has both the Running and Stopped Service.

Splunk detects both service status. The only issue is Spunk detects all services as the same value.

We do not want to use a csv file though. What the script does is it takes all the services, status and all details we require. Then it logs to a log file. We then have a .conf file that does the regex to parse the data.

So for example, in Splunk dashboard, it shows Service A = Running, Service B = Stopped. However, it cannot be identified in the Dashboard. The Dashboard displays both services with the same color.

Once again, thank you for your response 🙂

[gcusello]

Hi @annbingle,

please, share a sample of your events (with all the conditions) and the search you're using.

Ciao.

Giuseppe