All Apps and Add-ons

Why am I seeing CPU Usage discrepancy between S.o.S - Splunk on Splunk and PerfMon?

nk-1
Path Finder

Splunk-on-Splunk (SoS) indicates 80% - 100% CPU usage, due to searches:
SoS_Chart

PerfMon indicates about 25% CPU usage:
PerfMon_Chart

Why the discrepancy?

0 Karma
1 Solution

hexx
Splunk Employee
Splunk Employee

I'm not sure which perfmon counter you are querying, but my best guess here is that it is a counter that expresses CPU usage as a percentage of the total amount of CPU resources on the system.

In contrast, S.o.S expresses CPU usage per process class (the "searches" class including all search processes) as a percentage of one CPU core.

If my theory is correct, you have a 4-core machine here where one or two searches are running and consuming ~ 1 CPU cores altogether. Therefore, the CPU usage of searches can be expressed as 25% of all CPU resources (1 out of 4 available CPU cores) or ~ 1 CPU core / 100% of 1 CPU core.

Did I guess correctly?

View solution in original post

0 Karma

hexx
Splunk Employee
Splunk Employee

I'm not sure which perfmon counter you are querying, but my best guess here is that it is a counter that expresses CPU usage as a percentage of the total amount of CPU resources on the system.

In contrast, S.o.S expresses CPU usage per process class (the "searches" class including all search processes) as a percentage of one CPU core.

If my theory is correct, you have a 4-core machine here where one or two searches are running and consuming ~ 1 CPU cores altogether. Therefore, the CPU usage of searches can be expressed as 25% of all CPU resources (1 out of 4 available CPU cores) or ~ 1 CPU core / 100% of 1 CPU core.

Did I guess correctly?

0 Karma

nk-1
Path Finder

You are correct.
Thanks!
Splunk_Server_CPU

Using

[perfmon://CPU Load]
counters = % Processor Time;% User Time
instances = _Total
interval = 10
object = Processor
index = wmi

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...