Calculate total duration when many transactions are overlap in the time

[wcastillocruz]

Hello dear community,
help me on this issue please.
When using the concurrency command to find out if transactions overlap in time, and if so, is it possible to calculate the total duration of the incident taking the overlap into account.

for example :

transaction 1:
start -> 10 a.m.
end -> 11 a.m.

Transaction 2:
start -> 10:30 am
end -> 11:30 am

transaction 1 concerns process1 and transaction 2 concern porcess2 but the two transactions correspond to the same application X

before, to calculate the total duration of the incident on application X
I added the duration of transaction 1 + the duration of transaction 2.
this is the correct way when incidents (transactions) do not overlap, but when they overlap as in the previous example. the total incident duration of the application is equal to 1h30 and not to 2h.

using concurrency command can we calculate this duration?

[ITWhisperer]

Can you identify which message belongs to which transaction? If so, you can "group" the events by that identifier and determine your duration from that.

for example :

10 a.m transaction 1 start
10:30 a.m transaction 2 start
11 a.m transaction 1 end
11:30 a.m transaction 2 end

[wcastillocruz]

Hi @ITWhisperer,
Thank you for your answer.

yes, I can identify each transaction with a unique ID.
I'm already doing this. or do you mean to combine my two transactions into one?

[ITWhisperer]

How do you know that transaction 1 and transaction 2 are part of the same incident?

If I understand correctly, you want the time between first start (10 am) and last end (11:30 am), is that right?

If the first transaction finished at 11 am and the second one started at 12pm, both transactions taking an hour, would you want the duration to be 2 hours or 3 hours?

[wcastillocruz]

@ITWhisperer,
Thanks again for your reply.

well,
I identify each transaction thanks to a series of data that each transaction contains.
this is the start of my search:

index = index1
| eval ID = Service + "_" + Env + "_" + Apps + "_" + Function
| addinfo
| transaction ID startswith = (severity = 2) endswith = (severity = 0 OR severity = 1 OR severity = -1) maxevents = 4

"If I understand correctly, you want the time between first start (10 am) and last end (11:30 am), is that right?"
yes, at this time the duration of the incident for the Application is 1:30 hour.

but if
transaction 1 =
start 10:00 AM
end 11:00 AM

transaction 2 =
start 11:00 AM
end 12:00 PM

in this case the duration is equal to the sum of the duration transaction 1 + duration transaction 2
note: I have several Applications, and my ID allows me to separate transactions.

[ITWhisperer]

It is still not clear what ties transaction 1 to transaction 2. Suppose there was a third transaction (transaction 3) which was part of a different "incident". How would you know whether to consider the start of transaction 1 and the end of transaction 2, or the start of transaction 1 and the end of transaction 3?

[wcastillocruz]

Here is an example of two transactions and underlined in blue which allows me to differentiate them

[ITWhisperer]

index = index1
| eval IncidentID = Service + "_" + Env + "_" + Apps
| stats first(_time) as start last(_time) as end by IncidentID
| eval duration=end-start

[wcastillocruz]

@ITWhisperer 
it is not as easy as it may seem.
I have listed several months of events, this is not a specific case.
moreover I can index several events for the same application in a day, and it is possible that these events do not overlap and which do not follow one another.

by following your solution I get this:

while for application A for example, nothing happened between 1:00 p.m. and 2:00 p.m. and with your solution I add this time to my total incident duration

thank you very much anyway.

[ITWhisperer]

OK I got it - I had misread your first graphic. Yes, it is a bit more complicated.

[ITWhisperer · Accepted solution]

| makeresults | eval _raw="time,application,transaction,action
10:00,A,1,start
12:00,A,1,end
11:00,A,2,start
13:00,A,2,end
15:00,A,3,start
17:00,A,3,end
10:00,B,1,start
12:00,B,1,end
13:00,B,2,start
15:00,B,2,end
16:00,B,3,start
18:00,B,3,end"
| multikv forceheader=1
| fields - _time _raw linecount
| eval _time=strptime(strftime(now(),"%Y/%m/%d")."T".time.":00","%Y/%m/%dT%H:%M:%S")
| fields - time
| sort application _time
| eval process=if(action="start",1,-1)
| streamstats sum(process) as concurrent by application
| eval starttime=if(concurrent=1 AND process=1,_time,null)
| eval endtime=if(concurrent=0 AND process=-1,_time,null)
| filldown starttime
| eval duration=endtime-starttime
| stats sum(duration) as duration by application
| eval duration=tostring(duration,"duration")

[ITWhisperer]

Essentially, use streamstats to keep a running total of active processes, note when the first of an overlapping set starts and when the last of the overlapping set ends, then find the difference and add all the differences to give a total processing time

[wcastillocruz]

it's not you but me who explains badly

Thank a lot

[wcastillocruz]

that explains better than me 🙂 

[ITWhisperer]

That looks like you want the first message from Application A and the last message from Application A. Can an application handle more than one incident? Do you need to be able to distinguish between the incidents an application is handling? How would you do this?