Assume I have an input file like the following:
2015-07-28 12:00:01 Executing function a... 2015-07-28 12:00:02 debug1 2015-07-28 12:00:03 debug2 2015-07-28 12:00:04 Completing function a (value=-1) 2015-07-28 12:00:05 Executing function a... 2015-07-28 12:00:06 debug3 2015-07-28 12:00:07 debug4 2015-07-28 12:00:08 Completing function a (value=0)
I want to build a transaction object that begins with the first line and ends with the last. If I use
startswith="Executing function a..." and
endswith="Completing function a (value=0)", it appears that the second
Executing function a... evicts the first, even though an
endswith has not occurred. I end up with a single transaction that begins at 12:00:05 and ends at 12:00:08, and the earlier is an incomplete transaction.
Is there any way to get
transaction not to evict the previous transaction when it encounters another
startswith (prior to an
I'm not sure if thats the key but your
endswith="Completing function a (value=0)" is explicit looking for
It might be enough to simplify your search to something like:
... | transaction startswith="Executing" endswith="Completing"
If I do that, it results in two transactions, one from 12:00:01 - 12:00:04 and another from 12:00:05 - 12:00:08, which is not the desired outcome. The explicit "value=0" is requred.
Ah I see ... i've got that wrong. It's a tricky problem ...Im not sure if there is a simple way to not evict the first transaction.
If i get you right, you want all events from execution function a until Completing with value=0 in one transaction.It all depends on how your other events look like. Is it always function a? Are these events all in order? Maybe what you can try is to reduce your transaction to an
... | transaction endswith="(value=0)"
Answers to your questions:
Extract out the function name as a field, and use that in your transaction:
| rex "function\s(?<function>[^\s]+)\s" | transaction function startswith="Executing" endswith="(value=0)"
That should pull the whole shebang as a single transaction. However, this will omit any lines which do not have a 'function' field. (which may be context you need)
Make sure that 'value' field is extracted, then try the search suggested by @hgrow and filter the incomplete result by checking "| where value=0". This will drop the transaction with value=-1 (incomplete)
It's not the endswith that is the problem - I can successfully filter out the incorrect "ends". I can't filter out the incorrect "starts" because they are identical to each other. I need to somehow tell
transaction to ignore repeated starts.
If the logging is single-threaded (meaning you won't have processes interleaved with each other) then you can actually omit the 'startswith' and get what you want. Or, I do in testing at least.
If you may have multiple hosts running concurrently, then you want to include that in the transaction:
| transaction host endswith="(value=0)"
Ahh...yes...that would work, if there weren't "noise" between functions....I'll update the example to reflect that noise.