Splunk Search

How to run comparison after a group by?


I am running a query where I'm trying to calculate the difference between the start and end times a request travels through a service (aka latency). In order to achieve this I search for two logs: one for the start, one for the end, I then subtract the start and end times, and finally do a group by X_Request_ID-which is unique per request. What I have at this point is:

Screen Shot 2022-09-15 at 6.28.02 PM.png

What I want to do now is to only display the count of all requests that took over 1 second. 

My attempt at this looks like:

index=prod component="card-notification-service" eventCategory=transactions eventType=auth AND ("is going to process" OR ("to POST https://apay-partner-api.apple.com/ccs/v1/users/eventNotification/transactions/auth" AND status=204))

| eval diff=if(searchmatch("is going to process"), _time*-1, 0)

| eval Start=if(searchmatch("is going to process"), _time, NULL)

| eval diff=if(searchmatch("to POST https://app.transactions/auth"), diff+_time, diff)

| eval End=if(searchmatch("to POST https://app.transactions/auth"), _time, NULL) | eval seriesName="Baxter<->Saturn

| streamstats sum(diff) by X_Request_ID as FinalDiff |where FinalDiff> 1.0 

| timechart span=5m partial=f count by seriesName

I’ve gotten everything to compile fine before the bolded where clause above. I suspect it’s because in the streamstats command prior, the “as” is only naming the query and not persisting the grouping of the query. Regardless this leads me to the question I am trying to solve: How can I persist sum(diff) after grouping it by X_Request_ID so that in the next pipe I can perform a comparison in the where operation?

Labels (1)
0 Karma

Path Finder

Switch up the order of things a bit, and this should work for you:

| streamstats sum(diff) as FinalDiff by X_Request_ID |
where FinalDiff> 1.0 

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In the last month, the Splunk Threat Research Team (STRT) has had 2 releases of new security content via the ...

Announcing the 1st Round Champion’s Tribute Winners of the Great Resilience Quest

We are happy to announce the 20 lucky questers who are selected to be the first round of Champion's Tribute ...

We’ve Got Education Validation!

Are you feeling it? All the career-boosting benefits of up-skilling with Splunk? It’s not just a feeling, it's ...