All Apps and Add-ons

Why am I seeing CPU Usage discrepancy between S.o.S - Splunk on Splunk and PerfMon?

nk-1
Path Finder

Splunk-on-Splunk (SoS) indicates 80% - 100% CPU usage, due to searches:
SoS_Chart

PerfMon indicates about 25% CPU usage:
PerfMon_Chart

Why the discrepancy?

0 Karma
1 Solution

hexx
Splunk Employee
Splunk Employee

I'm not sure which perfmon counter you are querying, but my best guess here is that it is a counter that expresses CPU usage as a percentage of the total amount of CPU resources on the system.

In contrast, S.o.S expresses CPU usage per process class (the "searches" class including all search processes) as a percentage of one CPU core.

If my theory is correct, you have a 4-core machine here where one or two searches are running and consuming ~ 1 CPU cores altogether. Therefore, the CPU usage of searches can be expressed as 25% of all CPU resources (1 out of 4 available CPU cores) or ~ 1 CPU core / 100% of 1 CPU core.

Did I guess correctly?

View solution in original post

0 Karma

hexx
Splunk Employee
Splunk Employee

I'm not sure which perfmon counter you are querying, but my best guess here is that it is a counter that expresses CPU usage as a percentage of the total amount of CPU resources on the system.

In contrast, S.o.S expresses CPU usage per process class (the "searches" class including all search processes) as a percentage of one CPU core.

If my theory is correct, you have a 4-core machine here where one or two searches are running and consuming ~ 1 CPU cores altogether. Therefore, the CPU usage of searches can be expressed as 25% of all CPU resources (1 out of 4 available CPU cores) or ~ 1 CPU core / 100% of 1 CPU core.

Did I guess correctly?

0 Karma

nk-1
Path Finder

You are correct.
Thanks!
Splunk_Server_CPU

Using

[perfmon://CPU Load]
counters = % Processor Time;% User Time
instances = _Total
interval = 10
object = Processor
index = wmi

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...