I'm using the *NIX add-on for Splunk.
We receive "TOP.sh" information into Splunk. Top provides the process information and pctCPU.
I'm trying to run a statistics table search to do the following:
Pull the TOP data for a specific host.
Return the COMMAND (Process Name) and pctCPU.
Sum the pctCPU for all similar process names.
Create a new column with the total pctCPU for every process.
Divide the pctCPU total for each process by the total pctCPU of the box and put that in a separate field (cpuShare).
Return the process names and cpuShare.
I have a piechart that returns the data and creates the slices. But the statistical format of that doesn't have the percentages. Is there a hidden field I can display or do I need to do the division myself? This is going in a dashboard. I have obscured the real host name with $host_name$
Example of pie chart:
index=os host=$host_name$ source="top" COMMAND=* | table _time, host, USER, COMMAND, PID, pctCPU | chart sum(pctCPU) over COMMAND
Example of statistical table to evaluate the CPU share:
index=os host=$host_name$ source="top" COMMAND=* | table COMMAND, pctCPU | stats sum(pctCPU) by COMMAND | sort - sum(pctCPU) | eval bretCPU = [ search index=os host=$host_name$ source="top" COMMAND=* | table pctCPU | stats sum(pctCPU) as totalPctCPU | return totalPctCPU]
When I do a
pie chart it automatically calculates the percentages and I can see them in the
tooltip when I hover over a pie segment. If you need to see them more obviously and all together, then you can calculate them yourself and add them to the name field like this:
index=_* | stats count BY component | eventstats sum(count) AS total_count | eval pct = round((100 * count / total_count), 3) | eval component = component . "(" . pct . "%)" | table component count
I agree. The pie chart shows it. I want the equivalent data in a table. I'll try your logic and report back.
That works for pct of count, but that's not what I'm looking for. Let me show the data it might make more sense.
So the TOP.sh script (*NIX Add-on) runs every 1 minute and returns the processes running and the pctCPU. In our server, we have 44 cores, so you will see process grab 20 or 30 cores at times.
Time Interval #1
Time Interval #2
I wrote a chart search to sum by each process, which combines looks like this:
Splunk creates the piechart and that works fine. It also creates the pie slice percentages. I want this in a statistics table.
So, all I need to do is add a new field to piechart results that totals all of the process percentages together and then divided each process by the total in a new column. Then, return the Process and PCT.
COMMAND pctCPU totalCPU totalPCT
processA 535.6 4056.6 13.2
processB 116.2 4056.6 2.8
processC 3404.8 4056.6 83.9
I just keep getting a logic error with the syntax above. My subsearch works fine by itself. When I put it in the EVAL, it appears to return a boolean value. The search by itself returns a number.
| makeresults | eval raw="COMMAND=processA,pctCPU=103.8 COMMAND=processB,pctCPU=27.1 COMMAND=processC,pctCPU=3103.9:::COMMAND=processA,pctCPU=431.8 COMMAND=processB,pctCPU=89.1 COMMAND=processC,pctCPU=300.9" | makemv delim=":::" raw | mvexpand raw | makemv delim=" " raw | mvexpand raw | rename raw AS _raw | kv | rename COMMENT AS "Everything above generates sample event data; everything below is your solution" | chart sum(pctCPU) AS pctCPU OVER COMMAND | eventstats sum(pctCPU) AS pctCPUtotal | eval pctCPUpct = round((100 * pctCPU / pctCPUtotal), 3)