Splunk Search

Using top with Multiple Fields

David
Splunk Employee
Splunk Employee

I feel like there should be an easy answer for this, but that my brain isn't finding it, so hopefully someone can reprieve me.

Suppose I have a log with the processing time for a number of URLs, across a number of servers. I want to toss into a summary index the top 10 longest running URLs per server, so I can later use it in a subsearch for host=foo.

In essence, this could work if top supported it:

MySearch earliest=-1d@d latest=@d | bucket _time span=1d | stats sum(ProcessTime) as ProcessTime by URL, host | top limit=10 labelField=URL ProcessTime by host | stats values(URL) by host

This also feels like something that could work if stats supported it:

MySearch earliest=-1d@d latest=@d | bucket _time span=1d | stats limit=10 sum(ProcessTime) as ProcessTime by URL, host | stats values(URL) by host

How can I do what I'm trying to do?

0 Karma
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

Well, first of all, I will note that if you're using a summary, you should be aware that your daily summary won't aggregate, i.e., having the top 10 for each day in your summary does not let you get the top 10 for, say, a whole week in general.

If you just want what you're asking for, though, a quick way to get this is:

MySearch earliest=-1d@d latest=@d 
| bucket _time span=1d 
| stats sum(ProcessTime) as ProcessTime by URL,host
| sort host,-ProcessTime 
| streamstats global=f current=f window=0
    count by host
| where count < 10
| fields - count

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

Well, first of all, I will note that if you're using a summary, you should be aware that your daily summary won't aggregate, i.e., having the top 10 for each day in your summary does not let you get the top 10 for, say, a whole week in general.

If you just want what you're asking for, though, a quick way to get this is:

MySearch earliest=-1d@d latest=@d 
| bucket _time span=1d 
| stats sum(ProcessTime) as ProcessTime by URL,host
| sort host,-ProcessTime 
| streamstats global=f current=f window=0
    count by host
| where count < 10
| fields - count

David
Splunk Employee
Splunk Employee
0 Karma

David
Splunk Employee
Splunk Employee

I'm working on a different scenario for the same issue now, with much greater field variability. What is the upper limit of how many values I can toss at sort | streamstats | where | fields before I start getting failures?

I'm splitting by three fields -- FieldA has 30 options, FieldB has up to 2000 and FieldC has up to 10,000. In the raw data, right now I have about 500,000 different possibilities going into the sort, with the expectation of exceeding 1,000,000 during the lifetime of the app.

0 Karma
Get Updates on the Splunk Community!

New Case Study Shows the Value of Partnering with Splunk Academic Alliance

The University of Nevada, Las Vegas (UNLV) is another premier research institution helping to shape the next ...

How to Monitor Google Kubernetes Engine (GKE)

We’ve looked at how to integrate Kubernetes environments with Splunk Observability Cloud, but what about ...

Index This | How can you make 45 using only 4?

October 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...