topic Re: Calculate total duration when many transactions are overlap in the time in Splunk Search

Calculate total duration when many transactions are overlap in the time

wcastillocruz — Wed, 24 Mar 2021 11:50:06 GMT

Hello dear community,
help me on this issue please.
When using the concurrency command to find out if transactions overlap in time, and if so, is it possible to calculate the total duration of the incident taking the overlap into account.

for example :

transaction 1:
start -> 10 a.m.
end -> 11 a.m.

Transaction 2:
start -> 10:30 am
end -> 11:30 am

transaction 1 concerns process1 and transaction 2 concern porcess2 but the two transactions correspond to the same application X

before, to calculate the total duration of the incident on application X
I added the duration of transaction 1 + the duration of transaction 2.
this is the correct way when incidents (transactions) do not overlap, but when they overlap as in the previous example. the total incident duration of the application is equal to 1h30 and not to 2h.

using concurrency command can we calculate this duration?

Re: Calculate total duration when many transactions are overlap in the time

ITWhisperer — Wed, 24 Mar 2021 15:46:21 GMT

Can you identify which message belongs to which transaction? If so, you can "group" the events by that identifier and determine your duration from that.

for example :

10 a.m transaction 1 start
10:30 a.m transaction 2 start
11 a.m transaction 1 end
11:30 a.m transaction 2 end

Re: Calculate total duration when many transactions are overlap in the time

wcastillocruz — Thu, 25 Mar 2021 08:41:37 GMT

Hi @ITWhisperer,
Thank you for your answer.

yes, I can identify each transaction with a unique ID.
I'm already doing this. or do you mean to combine my two transactions into one?

Re: Calculate total duration when many transactions are overlap in the time

ITWhisperer — Thu, 25 Mar 2021 08:51:32 GMT

How do you know that transaction 1 and transaction 2 are part of the same incident?

If I understand correctly, you want the time between first start (10 am) and last end (11:30 am), is that right?

If the first transaction finished at 11 am and the second one started at 12pm, both transactions taking an hour, would you want the duration to be 2 hours or 3 hours?

Re: Calculate total duration when many transactions are overlap in the time

wcastillocruz — Thu, 25 Mar 2021 09:16:11 GMT

@ITWhisperer,
Thanks again for your reply.

well,
I identify each transaction thanks to a series of data that each transaction contains.
this is the start of my search:

index = index1
| eval ID = Service + "_" + Env + "_" + Apps + "_" + Function
| addinfo
| transaction ID startswith = (severity = 2) endswith = (severity = 0 OR severity = 1 OR severity = -1) maxevents = 4

"If I understand correctly, you want the time between first start (10 am) and last end (11:30 am), is that right?"
yes, at this time the duration of the incident for the Application is 1:30 hour.

but if
transaction 1 =
start 10:00 AM
end 11:00 AM

transaction 2 =
start 11:00 AM
end 12:00 PM

in this case the duration is equal to the sum of the duration transaction 1 + duration transaction 2
note: I have several Applications, and my ID allows me to separate transactions.

Re: Calculate total duration when many transactions are overlap in the time

wcastillocruz — Thu, 25 Mar 2021 09:25:13 GMT

that explains better than me 🙂

Re: Calculate total duration when many transactions are overlap in the time

ITWhisperer — Thu, 25 Mar 2021 09:26:26 GMT

It is still not clear what ties transaction 1 to transaction 2. Suppose there was a third transaction (transaction 3) which was part of a different "incident". How would you know whether to consider the start of transaction 1 and the end of transaction 2, or the start of transaction 1 and the end of transaction 3?

Re: Calculate total duration when many transactions are overlap in the time

ITWhisperer — Thu, 25 Mar 2021 09:33:53 GMT

That looks like you want the first message from Application A and the last message from Application A. Can an application handle more than one incident? Do you need to be able to distinguish between the incidents an application is handling? How would you do this?

Re: Calculate total duration when many transactions are overlap in the time

wcastillocruz — Thu, 25 Mar 2021 09:46:37 GMT

Here is an example of two transactions and underlined in blue which allows me to differentiate them

Re: Calculate total duration when many transactions are overlap in the time

ITWhisperer — Thu, 25 Mar 2021 09:51:47 GMT

index = index1 | eval IncidentID = Service + "_" + Env + "_" + Apps | stats first(_time) as start last(_time) as end by IncidentID | eval duration=end-start

Re: Calculate total duration when many transactions are overlap in the time

wcastillocruz — Thu, 25 Mar 2021 10:29:49 GMT

@ITWhisperer
it is not as easy as it may seem.
I have listed several months of events, this is not a specific case.
moreover I can index several events for the same application in a day, and it is possible that these events do not overlap and which do not follow one another.

by following your solution I get this:

while for application A for example, nothing happened between 1:00 p.m. and 2:00 p.m. and with your solution I add this time to my total incident duration

thank you very much anyway.

Re: Calculate total duration when many transactions are overlap in the time

ITWhisperer — Thu, 25 Mar 2021 10:46:59 GMT

OK I got it - I had misread your first graphic. Yes, it is a bit more complicated.

Re: Calculate total duration when many transactions are overlap in the time

wcastillocruz — Thu, 25 Mar 2021 11:06:02 GMT

it's not you but me who explains badly

Thank a lot

Re: Calculate total duration when many transactions are overlap in the time

ITWhisperer — Thu, 25 Mar 2021 11:11:03 GMT

| makeresults | eval _raw="time,application,transaction,action 10:00,A,1,start 12:00,A,1,end 11:00,A,2,start 13:00,A,2,end 15:00,A,3,start 17:00,A,3,end 10:00,B,1,start 12:00,B,1,end 13:00,B,2,start 15:00,B,2,end 16:00,B,3,start 18:00,B,3,end" | multikv forceheader=1 | fields - _time _raw linecount | eval _time=strptime(strftime(now(),"%Y/%m/%d")."T".time.":00","%Y/%m/%dT%H:%M:%S") | fields - time | sort application _time | eval process=if(action="start",1,-1) | streamstats sum(process) as concurrent by application | eval starttime=if(concurrent=1 AND process=1,_time,null) | eval endtime=if(concurrent=0 AND process=-1,_time,null) | filldown starttime | eval duration=endtime-starttime | stats sum(duration) as duration by application | eval duration=tostring(duration,"duration")

Re: Calculate total duration when many transactions are overlap in the time

ITWhisperer — Thu, 25 Mar 2021 11:13:48 GMT

Essentially, use streamstats to keep a running total of active processes, note when the first of an overlapping set starts and when the last of the overlapping set ends, then find the difference and add all the differences to give a total processing time