Splunk Search
Highlighted

define transaction and gather knowledge from it

Explorer

Hi have logs look likes below, and want to define where transaction begin and where finished.
for example at ID654321 begin "654321 start" and finish at "654321 Message Received".

expected output 1 (overall report):
number of success transaction 1 654321
number of fail transaction (start without finish) 1 123456

expected output 2 (calculate transaction delay):
654321 2 (m)

2020-01-19 13:20:15,093 INFO  ABC.InIT-AppName-123456 [Performance] start[123456789] time[34] tag[ok]
2020-01-19 13:20:15,093 INFO  ABC.InIT-AppName-123456 [Processor] AdministrationProcessor Accomplished: A[xxx] B[yyy] C[1A0000] DE[00] F[0000] GH[ABC.OutIT-AppName] Status[PERSIST-LOGOUT,BACKWARD]
2020-01-19 13:20:15,099 INFO  ABC.InIT-AppName-123456 [ProcessorService] Message Processed: M[xxx] T[yyy] C[1A0000] DE[00] F[0000] GH[ABC.OutIT-AppName] Status[EXIST-LOGOUT,BACKWARD]
2020-01-19 13:20:15,099 INFO  ABC.InIT-AppName-123456 [Performance] start[123456789] time[313] tag[process]
2020-01-19 13:20:15,110 INFO  ABC.InIT-AppName-123456 [Manager] Send Message [123456789A123456789*] to [ABC.app.AppName] 
2020-01-19 13:20:00,114 INFO  ABC.InIT-AppName-654321 [Performance] start[987654321] time[7] tag[send]
2020-01-19 13:20:08,181 INFO  ABC.InIT-AppName-654321 [Listener] Receive Message[987654321B123456789*] from [ABC.AppName.app] 
2020-01-19 13:22:00,185 INFO  ABC.InIT-AppName-654321 [ProcessorService] Normal Message Received: A[111] B[222] NM[123456789]

Any recommendation?
Thanks,

0 Karma
Highlighted

Re: define transaction and gather knowledge from it

Motivator

Hello @mehrdad_2000 ,

does the transaction 123456 contains two "starts"? If this is just a typo and every transaction ends with "Message Processed" then try this SPL:

first:

... | rex "InIT-AppName-(?<transID>\d+)" | transaction transID startswith=(start) endswith=("Message Received") keepevicted=1 | eval txn_status=if(closed_txn=1,"successful", "failed") | stats count, values(transID) by txn_status

second:

... | rex "InIT-AppName-(?<transID>\d+)" | transaction transID startswith=(start) endswith=("Message Received") keepevicted=1  | table transID duration | sort - duration

but if your log contains two transactions (and not three), then the SPL need to be modified

0 Karma
Highlighted

Re: define transaction and gather knowledge from it

Explorer

this part of logs variety
ABC.InIT-AppName-123456

for e.g. CDE.InOT-AppName1O-123456 [
or
FGHIJ.SMnOT-HA-AppName4O-123456 [

instead of (rex "InIT-AppName-(?\d+)") how can I extract id with variety pattern? (start after dash "-" , end before space bracket "["

0 Karma
Highlighted

Re: define transaction and gather knowledge from it

Motivator

Hi @mehrdad_2000,

try to replace letters with [a-zA-Z]{3,5}, dot with ., the dash with - in the regex. You can test your regex on https://regex101.com/

Good Luck!

0 Karma
Highlighted

Re: define transaction and gather knowledge from it

Explorer

Is it possible to write regex that consider only number between after”-“ before “[“.
I mean without define every elements that line start with?
I try different regex on https://regex101.com/ but not succeeded!

0 Karma
Highlighted

Re: define transaction and gather knowledge from it

Motivator

Hello @mehrdad_2000,

sure, it is possible. But the shorter the regex the higher the probability to catch wrong pattern.
I'd use this pattern (replace the numbers if needed):

  rex "[A-Z]{2,4}\.[a-zA-Z]{2,4}\-\w{2,10}\-(?<transID>\d+)\s+\[[A-Z]\w+\]"

check it here: https://regex101.com/r/oRpkAx/1

please accept the answer if it solves your query

0 Karma
Highlighted

Re: define transaction and gather knowledge from it

Explorer

This work perfectly for me.
Thank you 🙂

0 Karma