Trying to create a useful CPU usage search, but coming up short. This search does not reflect what is actually being used for some reason:
index=perfstats host=servername | timechart avg("%_Privileged_Time") by host
For example, when I compare to what the server is actually using to what the search is reporting, the search is coming back extremely low and doesn't match up with actual usage.
timechart will do aggregation on the event values, if you don't need/want that to happen use
chart like this:
index=perfstats host=servername | stats avg("%_Privileged_Time") AS avg_PrivTime max("%_Privileged_Time") AS max_PrivTime min("%_Privileged_Time") AS min_PrivTime by host, _time
index=perfstats host=servername | chart avg("%_Privileged_Time") AS avg_PrivTime max("%_Privileged_Time") AS max_PrivTime min("%_Privileged_Time") AS min_PrivTime over _time by host
Both results are basically the same like using timechart, but without the aggregation.
hope this helps ...
many thanks for this and it's a big help. I'm able to use this search to effectively track the average CPU consumption for the servers, but when I say trend it out over time (7+ days for example), it doesn't look very clean as in the trend lines are actually lines, they appear more as bars.
search - index=perfstats host=servername | chart avg("%_Processor_Time") over _time by host
I'll post an image..
sure this is why
chartdoes not aggregate like
timechart does and as another pointer, did your try other formats for your chart? You can also change the behavior of the lines for example no results (connect, treat as zero ...)
Almost there I think.
So, I adjusted the visualization (format, connect the dots basically), but it's weird as there is what looks to be a true CPU metric followed by a very low, inaccurate metric which makes the trend line extremely spiky and doesn't really resemble what the CPU is doing.
So, the search is pretty simple - index=perfstats host=eivc201vwin | chart avg("%_Processor_Time") over _time by host
This image shows how spiky the trend line is - http://pasteboard.co/2mtXDwFl.png
This image shows the actual sample data where you see this "piggy back" type metric - http://pbrd.co/16bamV9
You'll see a reading of say 45 followed immediately by a real low reading of 3.
Thanks again for your time!
It is grabbing the stats every 5 minutes, but it's weird as there are two stats for every 5 minute pull. They are a second apart and these are the metrics that appear to be making the trend spiky as the second metric is always much lower than the first. For example, a metric comes in at say 9:01:25 for 45% (or 45.87326) and then a metric comes in at 9:01:26 for say 2% (or 2.34344) and then the cycle starts again at 9:06:25.