Splunk Search

timechart limit: pick top 10 series with the highest peaks (of all time), not total sums

amomchilov
Explorer

I'm looking to investigate IP addresses with highest peak loads on our service. Here's my current query:

application="my-app" index="my-index" request client_ip="*" user_agent="*" request="*" kube_pod="web-*"
| timechart limit=10 useother=f span="5minute" count by client_ip

It works, and it's almost what I'm looking for. The documentation states:

If a single aggregation is specified, the score is based on the sum of the values in the aggregation for that split-by value. For example, for timechart avg(foo) BY <field> the avg(foo) values are added up for each value of to determine the scores.

If I understand this correctly, timeseries is picking the top 10 series whose sum of counts over the time span are the greatest. That is to say, it's picking the 10 top series by greatest integral.

Instead, I want to select the 10 top series with the highest peak values (of any time in the timespan). For example, if there's a single data point that shows 10,000 requests per second for a single second, I want to be chosen over another series that shows 10 requests/second for months straight (whose integral would be much greater, but with a much lower max peak).

Could you please help me do that?

0 Karma
1 Solution

to4kawa
Ultra Champion
...
| timechart useother=f span=5m count by client_ip where sum in top10

try where clause.

View solution in original post

to4kawa
Ultra Champion
...
| timechart useother=f span=5m count by client_ip where sum in top10

try where clause.

amomchilov
Explorer

Ah, I hadn't looked at that. From the docs it seems promising:

Specifies the criteria for including particular data series when a field is given in the . The most common use of this option is to look for spikes in your data rather than overall mass of distribution in series selection. The default value finds the top ten series by area under the curve. Alternately one could replace sum with max to find the series with the ten highest spikes. Essentially the default is the same as specifying where sum in top10. The has no relation to the where command.

I'll give it a shot and report back. How do where and limit interact?

0 Karma

to4kawa
Ultra Champion

.... where max in top10
It was this.
limithandle the count of result.
where handle the result.
I think so.

0 Karma

amomchilov
Explorer

Woo, that worked great!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...