Getting Data In

Combine multiple index searches into one overall stat

danroberts
Explorer

Hello, 

I'm trying to create a RAG dashboard that will show different colours should an issue occur with a service e.g. if a service stops working the stat would show as one and the colour would turn red, I can do this but what I am struggling with is combining multiple index searches into one overall stat e.g. index "windows_perfmon" disk runs out of space, stat increases to 1, a winhostmon index service stops and that stat increases to one, I'm struggling to combine these into one overall stat which would be 2 in this example. 

The current search I am using is: 

(index=winhostmon host="Splunktest" "Type=Service" sourcetype=WinHostMon DisplayName="Print Spooler" OR DisplayName="Snow Inventory Agent" StartMode="Auto" State="Stopped") OR
(index="windows_perfmon" host="Splunktest" object="LogicalDisk" counter="% Free Space" OR counter="Free Megabytes")
| eval diskInfoA = if(counter=="% Free Space",mvzip(instance,Value),null())
| eval diskInfoA1 = if(isnotnull(diskInfoA),mvzip(diskInfoA,counter),null())
| eval diskInfoB = if(counter=="Free Megabytes",mvzip(instance,Value),null())
| eval diskInfoB1 = if(isnotnull(diskInfoB),mvzip(diskInfoB,counter),null())
| stats list(diskInfoA1) AS "diskInfoA1", list(diskInfoB1) AS "diskInfoB1" by host, instance, _time
| makemv diskInfoA1 delim=","
| makemv diskInfoB1 delim=","
| eval freePerc = mvindex(diskInfoA1,1)
| eval freeMB = mvindex(diskInfoB1,1)
| eval usage=round(100-freePerc,2)
| eval GB = round(freeMB/1024,2)
| eval totalDiskGB = GB/(freePerc/100)
| stats max(usage) AS "Disk Usage", max(GB) AS "Disk Free", max(totalDiskGB) AS "Total Disk Size (GB)" by host instance
| where not instance="_Total"
| where NOT LIKE(instance,"%Hard%")
| search "Disk Usage" >90
| stats count

The result I get is just count=1 

Note in the above example I have stopped the print spooler on the server so the event count should be 2 as there is a disk that is also running above 90%

I have also tried the append version but again I cannot get it to combine the results.

index=winhostmon host="Splunktest" "Type=Service" sourcetype=WinHostMon DisplayName="Print Spooler" OR DisplayName="Snow Inventory Agent" StartMode="Auto" State="Stopped" | stats count|rename count as Service

|append [ search index="windows_perfmon" host="Splunktest" object="LogicalDisk" counter="% Free Space" OR counter="Free Megabytes"

| eval diskInfoA = if(counter=="% Free Space",mvzip(instance,Value),null())

| eval diskInfoA1 = if(isnotnull(diskInfoA),mvzip(diskInfoA,counter),null())

| eval diskInfoB = if(counter=="Free Megabytes",mvzip(instance,Value),null())

| eval diskInfoB1 = if(isnotnull(diskInfoB),mvzip(diskInfoB,counter),null())

| stats list(diskInfoA1) AS "diskInfoA1", list(diskInfoB1) AS "diskInfoB1" by host, instance, _time

| makemv diskInfoA1 delim=","

| makemv diskInfoB1 delim=","

| eval freePerc = mvindex(diskInfoA1,1)

| eval freeMB = mvindex(diskInfoB1,1)

| eval usage=round(100-freePerc,2)

| eval GB = round(freeMB/1024,2)

| eval totalDiskGB = GB/(freePerc/100)

| stats max(usage) AS "Disk Usage", max(GB) AS "Disk Free", max(totalDiskGB) AS "Total Disk Size (GB)" by host instance

| where not instance="_Total"

| where NOT LIKE(instance,"%Hard%")

| search "Disk Usage" >90

| stats count|rename count as Disk

]

The end goal of this is to just show one stat on a dashboard and when you click on that number it opens another dashboard that shows you the detail. 

 

Any help would be appreciated. 

Labels (2)
0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud's AI Assistant in Action Series: Auditing Compliance and ...

This is the third post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...