define transaction and gather knowledge from it

[indeed_2000]

Hi have logs look likes below, and want to define where transaction begin and where finished.
for example at ID654321 begin "654321 start" and finish at "654321 Message Received".

expected output 1 (overall report):
number of success transaction 1 654321
number of fail transaction (start without finish) 1 123456

expected output 2 (calculate transaction delay):
654321 2 (m)

2020-01-19 13:20:15,093 INFO  ABC.InIT-AppName-123456 [Performance] start[123456789] time[34] tag[ok]
2020-01-19 13:20:15,093 INFO  ABC.InIT-AppName-123456 [Processor] AdministrationProcessor Accomplished: A[xxx] B[yyy] C[1A0000] DE[00] F[0000] GH[ABC.OutIT-AppName] Status[PERSIST-LOGOUT,BACKWARD]
2020-01-19 13:20:15,099 INFO  ABC.InIT-AppName-123456 [ProcessorService] Message Processed: M[xxx] T[yyy] C[1A0000] DE[00] F[0000] GH[ABC.OutIT-AppName] Status[EXIST-LOGOUT,BACKWARD]
2020-01-19 13:20:15,099 INFO  ABC.InIT-AppName-123456 [Performance] start[123456789] time[313] tag[process]
2020-01-19 13:20:15,110 INFO  ABC.InIT-AppName-123456 [Manager] Send Message [123456789A123456789*] to [ABC.app.AppName] 
2020-01-19 13:20:00,114 INFO  ABC.InIT-AppName-654321 [Performance] start[987654321] time[7] tag[send]
2020-01-19 13:20:08,181 INFO  ABC.InIT-AppName-654321 [Listener] Receive Message[987654321B123456789*] from [ABC.AppName.app] 
2020-01-19 13:22:00,185 INFO  ABC.InIT-AppName-654321 [ProcessorService] Normal Message Received: A[111] B[222] NM[123456789]

Any recommendation?
Thanks,

[PavelP]

Hello @mehrdad_2000 ,

does the transaction 123456 contains two "starts"? If this is just a typo and every transaction ends with "Message Processed" then try this SPL:

first:

... | rex "InIT-AppName-(?<transID>\d+)" | transaction transID startswith=(start) endswith=("Message Received") keepevicted=1 | eval txn_status=if(closed_txn=1,"successful", "failed") | stats count, values(transID) by txn_status

second:

... | rex "InIT-AppName-(?<transID>\d+)" | transaction transID startswith=(start) endswith=("Message Received") keepevicted=1  | table transID duration | sort - duration

but if your log contains two transactions (and not three), then the SPL need to be modified

[indeed_2000]

this part of logs variety
ABC.InIT-AppName-123456

for e.g. CDE.InOT-AppName1O-123456 [
or
FGHIJ.SMnOT-HA-AppName4O-123456 [

instead of (rex "InIT-AppName-(?\d+)") how can I extract id with variety pattern? (start after dash "-" , end before space bracket "["

[PavelP]

Hi @mehrdad_2000,

try to replace letters with [a-zA-Z]{3,5}, dot with ., the dash with - in the regex. You can test your regex on https://regex101.com/

Good Luck!

[indeed_2000]

Is it possible to write regex that consider only number between after”-“ before “[“.
I mean without define every elements that line start with?
I try different regex on https://regex101.com/ but not succeeded!

[PavelP]

Hello @mehrdad_2000,

sure, it is possible. But the shorter the regex the higher the probability to catch wrong pattern.
I'd use this pattern (replace the numbers if needed):

  rex "[A-Z]{2,4}\.[a-zA-Z]{2,4}\-\w{2,10}\-(?<transID>\d+)\s+\[[A-Z]\w+\]"

check it here: https://regex101.com/r/oRpkAx/1

please accept the answer if it solves your query

[indeed_2000]

This work perfectly for me.
Thank you 🙂