Splunk Search

Multiline Interleaved Transactions

chiu_szeto_jpmc
Engager

I have a log file that contains multiple transactions. These transactions can span multiple lines. Since this is a multithreaded application, the transactions recorded for each user is interleaved. Fortunately, there is a thread id that I can use to segment each transaction.

How do I generate a transaction record for each user transaction?

Sample Data:


2013-09-15 08:09:49,371 127.0.0.1-exec-12 INFO UserId: abc123
2013-09-15 08:09:49,372 127.0.0.1-exec-12 INFO UserPartLookup?itemId=568
2013-09-15 08:09:49,387 127.0.0.1-exec-17 INFO UserId: xyz678
2013-09-15 08:09:49,392 127.0.0.1-exec-12 INFO LookupResultsComplete
2013-09-15 08:09:49,395 127.0.0.1-exec-17 INFO UserOrder?itemId=568
2013-09-15 08:09:49,392 127.0.0.1-exec-17 INFO LookupResultsComplete

Output Should be:


2013-09-15 08:09:49,371 127.0.0.1-exec-12 INFO UserId: abc123
2013-09-15 08:09:49,372 127.0.0.1-exec-12 INFO UserPartLookup?itemId=568
2013-09-15 08:09:49,392 127.0.0.1-exec-12 INFO LookupResultsComplete

2013-09-15 08:09:49,387 127.0.0.1-exec-17 INFO UserId: xyz678
2013-09-15 08:09:49,395 127.0.0.1-exec-17 INFO UserOrder?itemId=568
2013-09-15 08:09:49,392 127.0.0.1-exec-17 INFO LookupResultsComplete

My guess is the search should be something like: | rex field=_raw "^.+exec-(?\d{2})" | transaction startswith="UserId:" endswith="LookupResultsComplete"

Thanks.

Tags (1)
0 Karma

chiu_szeto_jpmc
Engager

Figured it out. I was missing the field-list. The answer is:


| rex field=_raw "^.+exec-(?d{2})" | transaction thr_id startswith="UserId:" endswith="LookupResultsComplete"

Thanks.

Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...