Splunk Search

Windows CPU by Process - How do I divide field value #1 by field value #2?

mightaswelby
Explorer

I am trying to get a representation of the percentage of CPU used per windows process based on the amount of processors per client server. To accomplish this I am looking to divide one field value (process_cpu_used_percent) by another field value (NumberOfLogicalProcessors) to get the final percentage of cpu used by process (based on the number of processors). I have written the following search to create the new field (cpu_by_process), however I do not see it listed in the Fields after I run the search. Should I be using a stats command instead?

Note:
Most fields are using the standard perfmon metrics from the Splunk Add-on for Microsoft Windows (https://splunkbase.splunk.com/app/742/) I've created a Powershell script that populates Splunk with the number of Logical Processors per client server to get this baseline, with a field name of NumberOfLogicalProcessors. Below is the search I have created, however the eval does not seem to be working properly by dividing the process CPU used by the number of processors per client server.

index="perfmon" (sourcetype=Perfmon:Process OR sourcetype=Perfmon:CPU counter="% Processor Time") OR sourcetype=Script:ProcessorCount host=LTJ09OKDU 
| eval cpu_by_process = process_cpu_used_percent / NumberOfLogicalProcessors
0 Karma
1 Solution

mightaswelby
Explorer

I was able to get the results need doing a subsearch.

index="perfmon" (sourcetype=Perfmon:Process OR sourcetype=Perfmon:CPU counter="% Processor Time")
| eval cpu_by_process=process_cpu_used_percent / [search sourcetype="Script:ProcessorCount" host="*"
| return $NumberOfLogicalProcessors]

View solution in original post

0 Karma

mightaswelby
Explorer

I was able to get the results need doing a subsearch.

index="perfmon" (sourcetype=Perfmon:Process OR sourcetype=Perfmon:CPU counter="% Processor Time")
| eval cpu_by_process=process_cpu_used_percent / [search sourcetype="Script:ProcessorCount" host="*"
| return $NumberOfLogicalProcessors]

0 Karma

MuS
Legend

Hi mightaswelby,

Just three quick things to check:

  1. Not all fields are listed by default, only fields with a coverage of more then 1% are shown - you need to change to all fields in the select fields view
  2. is NumberOfLogicalProcessors a string or a numeric value?
  3. the eval only works if the events have both fields NumberOfLogicalProcessors AND process_cpu_used_percent available

cheers, MuS

0 Karma

mightaswelby
Explorer

Thanks for the response MuS. Yes, I did verify the select all fields. The NumberOfLogicalProcessors is a numeric field, that is where I am running into the issue. This value will be dynamic based on the client server Processor Number count. (this works if I divide by a static number in the eval eg: | eval cpu_by_process = process_cpu_used_percent / 4 And when I run the search (without the eval) I am able to see both fields without issue, NumberOfLogicalProcessors AND process_cpu_used_percent. I'm just not sure how to approach this one.

0 Karma

MuS
Legend

I just had a poke in some perfmon events and was not able to find the field process_cpu_used_percent, where do you get this one from?

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...