All Apps and Add-ons

Why am I seeing CPU Usage discrepancy between S.o.S - Splunk on Splunk and PerfMon?

nk-1
Path Finder

Splunk-on-Splunk (SoS) indicates 80% - 100% CPU usage, due to searches:
SoS_Chart

PerfMon indicates about 25% CPU usage:
PerfMon_Chart

Why the discrepancy?

0 Karma
1 Solution

hexx
Splunk Employee
Splunk Employee

I'm not sure which perfmon counter you are querying, but my best guess here is that it is a counter that expresses CPU usage as a percentage of the total amount of CPU resources on the system.

In contrast, S.o.S expresses CPU usage per process class (the "searches" class including all search processes) as a percentage of one CPU core.

If my theory is correct, you have a 4-core machine here where one or two searches are running and consuming ~ 1 CPU cores altogether. Therefore, the CPU usage of searches can be expressed as 25% of all CPU resources (1 out of 4 available CPU cores) or ~ 1 CPU core / 100% of 1 CPU core.

Did I guess correctly?

View solution in original post

0 Karma

hexx
Splunk Employee
Splunk Employee

I'm not sure which perfmon counter you are querying, but my best guess here is that it is a counter that expresses CPU usage as a percentage of the total amount of CPU resources on the system.

In contrast, S.o.S expresses CPU usage per process class (the "searches" class including all search processes) as a percentage of one CPU core.

If my theory is correct, you have a 4-core machine here where one or two searches are running and consuming ~ 1 CPU cores altogether. Therefore, the CPU usage of searches can be expressed as 25% of all CPU resources (1 out of 4 available CPU cores) or ~ 1 CPU core / 100% of 1 CPU core.

Did I guess correctly?

0 Karma

nk-1
Path Finder

You are correct.
Thanks!
Splunk_Server_CPU

Using

[perfmon://CPU Load]
counters = % Processor Time;% User Time
instances = _Total
interval = 10
object = Processor
index = wmi

0 Karma
Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...