Filtering transaction content

[timpgray]

I have created a query that contains a transaction and it mostly works as expected. The unexpected part is that a couple of the returned transactions contain more than 500 lines and so is truncated. Further complicating the issue is the fact that the resulting lines that are kept are pretty much identical and don’t contain any information about the beginning or end of the transaction.

I thought I could just filter out the two lines that get repeated during that transaction before the transaction and I would be left with the more interesting content, but when I do this, the two transactions do not show up at all.

Here is a minimum representation of the initial query:
index = ZZZ | transaction startswith=”AAA” endswith=”BBB”

And this ends up with some problem results that contain the string “YYYY”(over the 500 limit) and is not really related to the transaction(it occurs inside it, but I am not interested in these events).

So I modify the query to filter out these events:
index = ZZZ | regex _raw != “YYYY” | transaction startswith=”AAA” endswith=”BBB”

I expected to have the same transactions minus the lines containing “YYYY”, but instead these problem transactions just do not show up at all.

I can’t help but think I am missing something important here, such as how the transactions are defined, when they are defined, etc.

[emiller42]

those may not be real transactions if they only contain the repeated content. By default, a transaction will only span 1000 events. (maxevents= parameter) So if you have a transaction that has 3000 events between the start and end, it will display as three transactions. (And each will be truncated in the display to 500 lines)

When you remove the junk lines, then the transaction is no longer split because of the event count, and you end up with fewer transactions.

This is further supported by your observation that the transactions that disappeared do not show the beginning or end of the transaction. If the display truncates down to 500 lines, it'll be the first 500 so you should see a legitimate start to the transaction.

To make sure you're not losing anything, you can do a search of JUST your start/end criteria and see if there are indeed legitimate transactions being dropped. Another option is to run the transaction with keeporphans=true to retain any records that aren't getting collected into a transaction for some reason.