Splunk Search

How to run comparison after a group by?

kimsej
Explorer

I am running a query where I'm trying to calculate the difference between the start and end times a request travels through a service (aka latency). In order to achieve this I search for two logs: one for the start, one for the end, I then subtract the start and end times, and finally do a group by X_Request_ID-which is unique per request. What I have at this point is:

Screen Shot 2022-09-15 at 6.28.02 PM.png

What I want to do now is to only display the count of all requests that took over 1 second. 

My attempt at this looks like:

index=prod component="card-notification-service" eventCategory=transactions eventType=auth AND ("is going to process" OR ("to POST https://apay-partner-api.apple.com/ccs/v1/users/eventNotification/transactions/auth" AND status=204))

| eval diff=if(searchmatch("is going to process"), _time*-1, 0)

| eval Start=if(searchmatch("is going to process"), _time, NULL)

| eval diff=if(searchmatch("to POST https://app.transactions/auth"), diff+_time, diff)

| eval End=if(searchmatch("to POST https://app.transactions/auth"), _time, NULL) | eval seriesName="Baxter<->Saturn

| streamstats sum(diff) by X_Request_ID as FinalDiff |where FinalDiff> 1.0 

| timechart span=5m partial=f count by seriesName

I’ve gotten everything to compile fine before the bolded where clause above. I suspect it’s because in the streamstats command prior, the “as” is only naming the query and not persisting the grouping of the query. Regardless this leads me to the question I am trying to solve: How can I persist sum(diff) after grouping it by X_Request_ID so that in the next pipe I can perform a comparison in the where operation?

Labels (1)
0 Karma

thesplunkmonkey
Path Finder

Switch up the order of things a bit, and this should work for you:

| streamstats sum(diff) as FinalDiff by X_Request_ID |
where FinalDiff> 1.0 

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In January, the Splunk Threat Research Team had one release of new security content via the Splunk ES Content ...

Expert Tips from Splunk Professional Services, Ensuring Compliance, and More New ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Observability Release Update: AI Assistant, AppD + Observability Cloud Integrations & ...

This month’s releases across the Splunk Observability portfolio deliver earlier detection and faster ...