Lets say I was running a search for an hour period from 10:00 until 11:00 and we had a particular transaction that consisted of 2 or more events - the first occurring at 09:59 and the last at 10:01.
Using the default Transaction command any events which occurred before 10:00 would not be included and we would therefore not be viewing the whole transaction. Likewise, if a transaction started at 10:59 and didn't end until 11:01, any events which occurred after 11:00 would be dropped.
Is there any way to include all events related to transactions which started or ended during the specified search time range?
Conversely, If this is not possible it would be helpful to drop any transactions which did not start and end within the time range - is there any way to achieve this?
1. No. Your search range determines the set of events which you will be processing (statsing, combining into transactions and so on). There is no way to "pull" additional events bar some ugly tricks like spawning a subsearch with map. But this is not something that I would recommend even considering.
2. In general, transaction is not a command which should be used lightly as it is relatively resource-intensive and has its limitations. If you can use fancy grouping and stats, do so.
3. If you use the transaction command, the _time of the combined transaction event is that of the earliest "source" event from this transaction. So you can filter on that field after combining your events. Alternatively, if you use stats instead, you can calculate earliest(_time) from your events group and filter on that.
Thank you very much for your very prompt response.
1. Good 🙂
2. Can't help you without a proper sample of events and description of desired outcome but it is possible that transaction is the only solution. It sometimes happens (the command is there for a reason after all ;-)). It's just that performance-wise it's often better to find another way for your search. Especially if it's gonna be invoked often and over a big set of data. If it's a one-off, not worth the effort probably.
3. For example something like that:
index=whatever other=conditions earliest=-2d@d latest=-1d@d
| <your evals, transactions and whatnot>
| where _time<relative_time(now(),"-1d@d-6h")
The last "where" command limits your results only to those that started not later than 6 hours before your "latest" parameter of the search. Which means that you're searching within a day - from -2d@d till -1d@d but later filter out 6 latest hours from this range
Thank you for taking the time to respond.
I tried various combinations of keeporphans and keepevicted but it didn't have any effect on transactions that ran over the search time range