topic How to chart raw Windows perfmon values over time? in Splunk Search

How to chart raw Windows perfmon values over time?

tmarlette — Fri, 23 Jan 2015 22:03:12 GMT

I'm attempting to chart some raw windows perfmon values on a chart over time, and I can't seem to find a way. I've been surfing here for a bit, but I haven't seen anything, so i figured I would ask.

This is the search that I used:

sourcetype=perfmon* collection=Process instance=splunkd counter="% Processor Time" | timechart avg(Value) by host

The thing is i'm not looking for an 'average' of the values, i'm just looking to chart the raw values over time. I don't need an avg / sum / max / min function, just how to plot the raw data.
I've attempted this as well:

sourcetype=perfmon* collection=Process instance=splunkd counter="% Processor Time" | timechart Value by host

but splunk says "Error in 'timechart' command: The specifier 'Value' is invalid. It must be in form (). For example: max(size)."

Is there a way to do this without the function?

Re: How to chart raw Windows perfmon values over time?

somesoni2 — Fri, 23 Jan 2015 22:22:07 GMT

Just try this

sourcetype=perfmon* collection=Process instance=splunkd counter="% Processor Time" | table _time host Value

There is a limit on no of points that can be displayed on the chart.

Also, try this

sourcetype=perfmon* collection=Process instance=splunkd counter="% Processor Time" | table _time host Value | xyseries _time host Value

Re: How to chart raw Windows perfmon values over time?

thomrs — Fri, 23 Jan 2015 23:05:06 GMT

You have to specify an aggregator or function with timechart.

Give xyseries that should do what you need.

look http://docs.splunk.com/Documentation/Splunk/6.2.1/SearchReference/Xyseries

Re: How to chart raw Windows perfmon values over time?

martin_mueller — Sun, 25 Jan 2015 02:16:27 GMT

Charting the values as they are using plain xyseries is a bit troublesome because you will not get a linear x-axis. If there's a gap in your data that gap will be invisible, if there's times with higher sampling frequency those times will be stretched.

If you're unhappy with | timechart avg(Value) by host because it flattens your data too much, consider increasing the number of data points like this: | timechart bins=500 avg(Value) by host
If you're unhappy with the average consider median(Value) or max(Value) or p95(Value) depending on your needs.

Re: How to chart raw Windows perfmon values over time?

tmarlette — Mon, 26 Jan 2015 17:37:33 GMT

xyseries did the trick, but ill keep using a bunch of bins in mind in order to limit the flattening of the data.

Thanks guys!

Re: How to chart raw Windows perfmon values over time?

ppablo — Tue, 27 Jan 2015 05:35:10 GMT

Hi @tmartlette

Glad you got a lot of input from other users to find your solution. Don't forget to accept one of the answers to resolve this post!

Patrick

Re: How to chart raw Windows perfmon values over time?

tmarlette — Tue, 27 Jan 2015 15:31:50 GMT

I would, but for some reason it won't let me at the moment. shrugs. This is the correct answer that I'm commenting on, though others have some good things to watch for as well.

Re: How to chart raw Windows perfmon values over time?

hagjos43 — Wed, 28 Jan 2015 18:43:57 GMT

This also works (add host values as needed):

 sourcetype="Perfmon:CPU Load" | bucket _time span=1m | stats avg(Value) by _time

And will plot an average line as well.

Source: http://gosplunk.com/average-cpu-usage/