Combine multiple index searches into one overall stat

danroberts — Tue, 28 Nov 2023 17:01:36 GMT

Hello,

I'm trying to create a RAG dashboard that will show different colours should an issue occur with a service e.g. if a service stops working the stat would show as one and the colour would turn red, I can do this but what I am struggling with is combining multiple index searches into one overall stat e.g. index "windows_perfmon" disk runs out of space, stat increases to 1, a winhostmon index service stops and that stat increases to one, I'm struggling to combine these into one overall stat which would be 2 in this example.

The current search I am using is:

(index=winhostmon host="Splunktest" "Type=Service" sourcetype=WinHostMon DisplayName="Print Spooler" OR DisplayName="Snow Inventory Agent" StartMode="Auto" State="Stopped") OR (index="windows_perfmon" host="Splunktest" object="LogicalDisk" counter="% Free Space" OR counter="Free Megabytes") | eval diskInfoA = if(counter=="% Free Space",mvzip(instance,Value),null()) | eval diskInfoA1 = if(isnotnull(diskInfoA),mvzip(diskInfoA,counter),null()) | eval diskInfoB = if(counter=="Free Megabytes",mvzip(instance,Value),null()) | eval diskInfoB1 = if(isnotnull(diskInfoB),mvzip(diskInfoB,counter),null()) | stats list(diskInfoA1) AS "diskInfoA1", list(diskInfoB1) AS "diskInfoB1" by host, instance, _time | makemv diskInfoA1 delim="," | makemv diskInfoB1 delim="," | eval freePerc = mvindex(diskInfoA1,1) | eval freeMB = mvindex(diskInfoB1,1) | eval usage=round(100-freePerc,2) | eval GB = round(freeMB/1024,2) | eval totalDiskGB = GB/(freePerc/100) | stats max(usage) AS "Disk Usage", max(GB) AS "Disk Free", max(totalDiskGB) AS "Total Disk Size (GB)" by host instance | where not instance="_Total" | where NOT LIKE(instance,"%Hard%") | search "Disk Usage" >90 | stats count

The result I get is just count=1

Note in the above example I have stopped the print spooler on the server so the event count should be 2 as there is a disk that is also running above 90%

I have also tried the append version but again I cannot get it to combine the results.

index=winhostmon host="Splunktest" "Type=Service" sourcetype=WinHostMon DisplayName="Print Spooler" OR DisplayName="Snow Inventory Agent" StartMode="Auto" State="Stopped" | stats count|rename count as Service |append [ search index="windows_perfmon" host="Splunktest" object="LogicalDisk" counter="% Free Space" OR counter="Free Megabytes" | eval diskInfoA = if(counter=="% Free Space",mvzip(instance,Value),null()) | eval diskInfoA1 = if(isnotnull(diskInfoA),mvzip(diskInfoA,counter),null()) | eval diskInfoB = if(counter=="Free Megabytes",mvzip(instance,Value),null()) | eval diskInfoB1 = if(isnotnull(diskInfoB),mvzip(diskInfoB,counter),null()) | stats list(diskInfoA1) AS "diskInfoA1", list(diskInfoB1) AS "diskInfoB1" by host, instance, _time | makemv diskInfoA1 delim="," | makemv diskInfoB1 delim="," | eval freePerc = mvindex(diskInfoA1,1) | eval freeMB = mvindex(diskInfoB1,1) | eval usage=round(100-freePerc,2) | eval GB = round(freeMB/1024,2) | eval totalDiskGB = GB/(freePerc/100) | stats max(usage) AS "Disk Usage", max(GB) AS "Disk Free", max(totalDiskGB) AS "Total Disk Size (GB)" by host instance | where not instance="_Total" | where NOT LIKE(instance,"%Hard%") | search "Disk Usage" >90 | stats count|rename count as Disk ]

The end goal of this is to just show one stat on a dashboard and when you click on that number it opens another dashboard that shows you the detail.

Any help would be appreciated.

topic Combine multiple index searches into one overall stat in Getting Data In

Combine multiple index searches into one overall stat