Assume I have an input file like the following:
2015-07-28 12:00:01 Executing function a...
2015-07-28 12:00:02 debug1
2015-07-28 12:00:03 debug2
2015-07-28 12:00:04 Completing function a (value=-1)
2015-07-28 12:00:05 Executing function a...
2015-07-28 12:00:06 debug3
2015-07-28 12:00:07 debug4
2015-07-28 12:00:08 Completing function a (value=0)
I want to build a transaction object that begins with the first line and ends with the last. If I use startswith="Executing function a..."
and endswith="Completing function a (value=0)"
, it appears that the second Executing function a...
evicts the first, even though an endswith
has not occurred. I end up with a single transaction that begins at 12:00:05 and ends at 12:00:08, and the earlier is an incomplete transaction.
Is there any way to get transaction
not to evict the previous transaction when it encounters another startswith
(prior to an endswith
)?
If the logging is single-threaded (meaning you won't have processes interleaved with each other) then you can actually omit the 'startswith' and get what you want. Or, I do in testing at least.
If you may have multiple hosts running concurrently, then you want to include that in the transaction:
| transaction host endswith="(value=0)"
Ahh...yes...that would work, if there weren't "noise" between functions....I'll update the example to reflect that noise.
Or...I would post the update, if I had enough karma....
Assume that there are more events before the "Executing..." and after the "Completing" that should be excluded.
Any way to filter out that noise? It might be a good idea to shape the initial search to only grab the lines you really care about from these transactions.
Make sure that 'value' field is extracted, then try the search suggested by @hgrow and filter the incomplete result by checking "| where value=0". This will drop the transaction with value=-1 (incomplete)
It's not the endswith that is the problem - I can successfully filter out the incorrect "ends". I can't filter out the incorrect "starts" because they are identical to each other. I need to somehow tell transaction
to ignore repeated starts.
Hi jswarren
I'm not sure if thats the key but your endswith="Completing function a (value=0)"
is explicit looking for value=0
.
It might be enough to simplify your search to something like:
... | transaction startswith="Executing" endswith="Completing"
sincerely
hgrow
If I do that, it results in two transactions, one from 12:00:01 - 12:00:04 and another from 12:00:05 - 12:00:08, which is not the desired outcome. The explicit "value=0" is requred.
Ah I see ... i've got that wrong. It's a tricky problem ...Im not sure if there is a simple way to not evict the first transaction.
If i get you right, you want all events from execution function a until Completing with value=0 in one transaction.It all depends on how your other events look like. Is it always function a? Are these events all in order? Maybe what you can try is to reduce your transaction to an endswith
.
Something like ... | transaction endswith="(value=0)"
Answers to your questions:
Extract out the function name as a field, and use that in your transaction:
| rex "function\s(?<function>[^\s]+)\s" | transaction function startswith="Executing" endswith="(value=0)"
That should pull the whole shebang as a single transaction. However, this will omit any lines which do not have a 'function' field. (which may be context you need)