Multiline Interleaved Transactions

chiu_szeto_jpmc — Mon, 28 Sep 2020 14:47:17 GMT

I have a log file that contains multiple transactions. These transactions can span multiple lines. Since this is a multithreaded application, the transactions recorded for each user is interleaved. Fortunately, there is a thread id that I can use to segment each transaction.

How do I generate a transaction record for each user transaction?

Sample Data:


2013-09-15 08:09:49,371 127.0.0.1-exec-12 INFO UserId: abc123
2013-09-15 08:09:49,372 127.0.0.1-exec-12 INFO UserPartLookup?itemId=568
2013-09-15 08:09:49,387 127.0.0.1-exec-17 INFO UserId: xyz678
2013-09-15 08:09:49,392 127.0.0.1-exec-12 INFO LookupResultsComplete
2013-09-15 08:09:49,395 127.0.0.1-exec-17 INFO UserOrder?itemId=568
2013-09-15 08:09:49,392 127.0.0.1-exec-17 INFO LookupResultsComplete

Output Should be:



2013-09-15 08:09:49,371 127.0.0.1-exec-12 INFO UserId: abc123

2013-09-15 08:09:49,372 127.0.0.1-exec-12 INFO UserPartLookup?itemId=568

2013-09-15 08:09:49,392 127.0.0.1-exec-12 INFO LookupResultsComplete

2013-09-15 08:09:49,387 127.0.0.1-exec-17 INFO UserId: xyz678
2013-09-15 08:09:49,395 127.0.0.1-exec-17 INFO UserOrder?itemId=568
2013-09-15 08:09:49,392 127.0.0.1-exec-17 INFO LookupResultsComplete

My guess is the search should be something like: | rex field=_raw "^.+exec-(?\d{2})" | transaction startswith="UserId:" endswith="LookupResultsComplete"

Thanks.

Re: Multiline Interleaved Transactions

chiu_szeto_jpmc — Mon, 16 Sep 2013 21:36:29 GMT

Figured it out. I was missing the field-list. The answer is:


| rex field=_raw "^.+exec-(?d{2})" | transaction thr_id startswith="UserId:" endswith="LookupResultsComplete"

Thanks.

topic Re: Multiline Interleaved Transactions in Splunk Search

Multiline Interleaved Transactions

Re: Multiline Interleaved Transactions