Splunk Search

How to chart raw Windows perfmon values over time?

Motivator

I'm attempting to chart some raw windows perfmon values on a chart over time, and I can't seem to find a way. I've been surfing here for a bit, but I haven't seen anything, so i figured I would ask.

This is the search that I used:

sourcetype=perfmon* collection=Process instance=splunkd counter="% Processor Time" | timechart avg(Value) by host

The thing is i'm not looking for an 'average' of the values, i'm just looking to chart the raw values over time. I don't need an avg / sum / max / min function, just how to plot the raw data.
I've attempted this as well:

sourcetype=perfmon* collection=Process instance=splunkd counter="% Processor Time" | timechart Value by host

but splunk says "Error in 'timechart' command: The specifier 'Value' is invalid. It must be in form (). For example: max(size)."

Is there a way to do this without the function?

0 Karma
1 Solution

SplunkTrust
SplunkTrust

Just try this

sourcetype=perfmon* collection=Process instance=splunkd counter="% Processor Time" | table _time host Value

There is a limit on no of points that can be displayed on the chart.

Also, try this

sourcetype=perfmon* collection=Process instance=splunkd counter="% Processor Time" | table _time host Value | xyseries _time host Value

View solution in original post

Contributor

This also works (add host values as needed):

 sourcetype="Perfmon:CPU Load" | bucket _time span=1m | stats avg(Value) by _time

And will plot an average line as well.

Source: http://gosplunk.com/average-cpu-usage/

0 Karma

SplunkTrust
SplunkTrust

Charting the values as they are using plain xyseries is a bit troublesome because you will not get a linear x-axis. If there's a gap in your data that gap will be invisible, if there's times with higher sampling frequency those times will be stretched.

If you're unhappy with | timechart avg(Value) by host because it flattens your data too much, consider increasing the number of data points like this: | timechart bins=500 avg(Value) by host
If you're unhappy with the average consider median(Value) or max(Value) or p95(Value) depending on your needs.

Communicator

You have to specify an aggregator or function with timechart.

Give xyseries that should do what you need.

look http://docs.splunk.com/Documentation/Splunk/6.2.1/SearchReference/Xyseries

SplunkTrust
SplunkTrust

Just try this

sourcetype=perfmon* collection=Process instance=splunkd counter="% Processor Time" | table _time host Value

There is a limit on no of points that can be displayed on the chart.

Also, try this

sourcetype=perfmon* collection=Process instance=splunkd counter="% Processor Time" | table _time host Value | xyseries _time host Value

View solution in original post

Motivator

xyseries did the trick, but ill keep using a bunch of bins in mind in order to limit the flattening of the data.

Thanks guys!

0 Karma

Community Manager
Community Manager

Hi @tmartlette

Glad you got a lot of input from other users to find your solution. Don't forget to accept one of the answers to resolve this post!

Patrick

0 Karma

Motivator

I would, but for some reason it won't let me at the moment. shrugs. This is the correct answer that I'm commenting on, though others have some good things to watch for as well.

0 Karma