Metrics search average and max

[HaakonRuud]

Hi guys! 

I've been struggeling for a while understanding metrics. When making a line chart for both average and max  value the trend is exact the same. This is the query:

| mstats avg("% Processor Time") as Avg, max("% Processor Time") as Max Where index="metric_index" AND collection=CPU AND host="host" span=1m
| fields _time, Avg, Max

But if I do avg and max of the value in the same time range I get two different values. Query used:

| mstats avg("% Processor Time") as Avg, max("% Processor Time") as Max Where index="metric_index" AND "collection"="CPU" AND "host"="host"

Earlier I had this data ingested as events and  then I had different trend for avg and max..

The inputs.conf file looks like this (using the Splunk_TA_windows app):
## CPU
[perfmon://CPU]
counters = % Processor Time
disabled = 0
samplingInterval = 2000
stats = average; min; max
instances = _Total
interval = 60
mode = single
object = Processor
useEnglishOnly=true
formatString = %.2f
index = metric_index


Are someone able to explain why this happens? Thanks in advance

[tscroggins · Accepted solution]

Hi @HaakonRuud,

When mode = single, it's implied the stats setting is ignored. As a result, the event will contain an average of samples collected every samplingInterval millseconds over interval seconds.

Your collection interval is 60 seconds (1 minute):

interval = 60

Your mstats time span is also 1 minute:

span=1m

As a result, you only have 1 event per time interval, so the mean and maximum will be equivalent.

[HaakonRuud]

Thank you so much for the explanation. This make so much sense when you describe it (and something I should be able to think of my self).