Splunk App/Add-on for Unix and Linux: Why do we sometimes get different results for CPU data between a Splunk search and sar output?

[gregbo]

I've installed Splunk App for *nix on my Splunk installation, and the add-on on a Linux VM. I want to know if I can trust the data I'm getting in Splunk, so I ran sar on the VM for a few minutes and tried to compare the output with data from Splunk ( sourcetype=cpu). Every once in a while, the data matches very closely, but a lot of other times they don't seem to even be close. Anybody know why?

[jcoates_splunk]

Are you running the same command at the same time? Take a look at cpu.sh, the if [ "x$KERNEL" = "xLinux" ] section