Getting Data In

Is there a problem with splunk getting data from the Perfmon Process counter?

gregbo
Communicator

Someone is telling me that Splunk doesn't grab Perfmon data properly...they are getting %Processor Time from both the Process and Processor counters. They say that if they grab the data from Process for each process and add them up, they don't get the same value as the value from the Processor counter (even taking the number of CPUs/Cores/Hyperthreading into account). They say they asked Splunk for updates and got some updates and they still don't work.

1 Solution

DavidHourani
Super Champion

Hi @gregbo,

It's true, perfmon has a lot of limitations when it comes to monitoring processes. Did some research and some of the limitations are :

  • Perfmon doesn't collect null values for CPU usage.
  • For multi-core Perfmon max CPU is 100% if the option useWinApiProcStats is not set. This option is best practice is case of multi-core systems as it uses theGetProcessTime function to build multi core CPU and Processor KPIs.
  • Window perf monitor only reads cooked data, so if you want more precise results you can follow the steps described here on this blog :https://robertlabrie.wordpress.com/2016/01/06/windows-cpu-monitoring-with-splunk/

Any limitations you are mentioning in your questions would be the same limitations for GetProcessTime() if you've activated useWinApiProcStats (which you should have since you're on a multi-core host 😞
https://docs.microsoft.com/en-us/windows/desktop/api/processthreadsapi/nf-processthreadsapi-getproce...

Let me know if that helps.

Cheers,
David

View solution in original post

DavidHourani
Super Champion

Hi @gregbo,

It's true, perfmon has a lot of limitations when it comes to monitoring processes. Did some research and some of the limitations are :

  • Perfmon doesn't collect null values for CPU usage.
  • For multi-core Perfmon max CPU is 100% if the option useWinApiProcStats is not set. This option is best practice is case of multi-core systems as it uses theGetProcessTime function to build multi core CPU and Processor KPIs.
  • Window perf monitor only reads cooked data, so if you want more precise results you can follow the steps described here on this blog :https://robertlabrie.wordpress.com/2016/01/06/windows-cpu-monitoring-with-splunk/

Any limitations you are mentioning in your questions would be the same limitations for GetProcessTime() if you've activated useWinApiProcStats (which you should have since you're on a multi-core host 😞
https://docs.microsoft.com/en-us/windows/desktop/api/processthreadsapi/nf-processthreadsapi-getproce...

Let me know if that helps.

Cheers,
David

Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...