I'm looking to investigate IP addresses with highest peak loads on our service. Here's my current query:
application="my-app" index="my-index" request client_ip="*" user_agent="*" request="*" kube_pod="web-*" | timechart limit=10 useother=f span="5minute" count by client_ip
It works, and it's almost what I'm looking for. The documentation states:
If a single aggregation is specified, the score is based on the sum of the values in the aggregation for that split-by value. For example, for
timechart avg(foo) BY <field>the
avg(foo)values are added up for each value of to determine the scores.
If I understand this correctly,
timeseries is picking the top
10 series whose sum of
counts over the time span are the greatest. That is to say, it's picking the
10 top series by greatest integral.
Instead, I want to select the
10 top series with the highest peak values (of any time in the timespan). For example, if there's a single data point that shows 10,000 requests per second for a single second, I want to be chosen over another series that shows 10 requests/second for months straight (whose integral would be much greater, but with a much lower max peak).
Could you please help me do that?
... | timechart useother=f span=5m count by client_ip where sum in top10
Ah, I hadn't looked at that. From the docs it seems promising:
Specifies the criteria for including particular data series when a field is given in the . The most common use of this option is to look for spikes in your data rather than overall mass of distribution in series selection. The default value finds the top ten series by area under the curve. Alternately one could replace sum with max to find the series with the ten highest spikes. Essentially the default is the same as specifying where sum in top10. The has no relation to the where command.
I'll give it a shot and report back. How do
.... where max in top10
It was this.
limithandle the count of result.
where handle the result.
I think so.