Compute some stats without discarding full results

[waisbrot]

I have logs of conversations, identified by a common field (a unique ID) and an end-marker. Some conversations get aborted without ending normally. I want a search that produces {_time, duration} pairs for every conversation that completed, and either {_time, running-sum} or final-sum for orphan transactions.

I could do this as two separate searches, one to get non-orphans

... | transaction convo_id endswith="end" unifyends=true keeporphans=false | table _time, duration

and one to get the orphans

... | transaction convo_id endswith="end" unifyends=true keeporphans=true | stats count(_txn_orphan) as orphan_count

But I'd rather not compute the transaction and the pipeline up to it twice in a row. Can I combine the last two parts of my search somehow?

[rsennett_splunk]

I was thinking that what you might want to do is something like what Sondra Russell is talking about in this blog: http://blogs.splunk.com/2014/01/23/quick-n-dirty-funnels/

This is referring to a different contextual set of data, however all you need is a UniqueID and an end marker... there are ways you can "fill in" missing field so that you can basically see if they are there or not, which helps with "state" which is what you're checking in order to correlate the events. Can you show us some sample data?

[somesoni2 · Accepted solution]

Try something like this

your base search...| eventstats count as TotalEvents | transaction convo_id endswith="end" unifyends=true keeporphans=false | eventstats sum(eventcount) as TotalEventsInTransaction | eval orphan_count=TotalEvents-TotalEventsInTransaction | table _time, duration,orphan_count