Transactions - grouping actions and matching on multiple fields

[thesteve]

I have a logfile which contains a set of performance related transactional data. I'm having trouble wrapping my brain around a proper search that will allow me to easily see transactions that are abnormally long.

Transactions are grouped together by name, thread id, and client-thread id (with a + or - indicating the start or end of the transaction)

Here's an example of what I'm looking at:

timestamp thread:123 client-thread:128 +login fred
timestamp thread:124 client-thread:132 +login lisa-hannigan
timestamp thread:123 client-thread:128 -login fred
timestamp thread:123 client-thread:127 +search fred searchstring
timestamp thread:145 client-thread: +flushcache flush system cache
timestamp thread:124 client-thread:132 +search lisa-hannigan searchstring
timestamp thread:126 client-thread:139 +search lisa-hannigan searchstring
timestamp thread:145 client-thread: -flushcache 
timestamp thread:123 client-thread:127 +search fred searchstring
timestamp thread:145 client-thread: +buildcache build system cache
timestamp thread:124 client-thread:132 -search lisa-hannigan searchstring
timestamp thread:126 client-thread:139 -search lisa-hannigan searchstring
timestamp thread:145 client-thread: -buildcache build system cache

I've built a regex that extracts the field data I need, but I'm having trouble wrapping my brain around a search string that will wrap the transactions appropriately. My "action" field contains +login/-login, +search/-search, etc.

I need to group these together: somehow pair up the +action and the -action, and I also need to match on both thread id and client thread id (which may be null).

My field data looks like:

threadid=123
clientthread=128
action=+login 

for the first line. I can adjust this if necessary. I'm wondering if this might be easier if I separate that out into action=login and actionstartend=+. I only show 4 action types here, but there are about 130 different actions, so I'd rather not have to do a search per action (i.e. login, search, flushcache, buildcache)

[martin_mueller · Accepted solution]

I think it'll be easier with the separated plus/minus, then you can do this:

... | transaction threadid clientthread action startswith="actionstartend=+" endswith="actionstartend=-"

If you can modify your log files I strongly recommend adding a unique transaction ID. Then you can simplify things and often also gain performance like this:

... | stats range(_time) as duration first(action) as action by transactionID