I have data as follows:
time=1 msgid=1 event=new_msg time=2 msgid=1 delivery=1 event=start_delivery time=3 delivery=1 event=deferred_delivery time=4 msgid=1 delivery=2 event=start_delivery time=5 delivery=2 event=successful_delivery time=6 msgid=1 event=end_msg
What I would like to achieve is to group events together from "new_msg" to "end_msg", including all "*_delivery" events.
I have tried to use
... | transaction msgid delivery startswith="new_msg" endswith="end_msg"
The problem is that I never get all the events together in one transaction, but mostly the events from time=1,2,3.
I also did some experiments with the "keepevicted", "keeporphans" and "connected" transaction parameters. Sometimes I also get the "final" events from time=4,5,6 as a separate transaction.
What never worked out is to have a single transaction for all of those events. Note that there may be more than just two delivery attempts than in the example.
My assumption is that "transaction" is unable to follow changing values in one of the provided fields, as it is the case with "delivery".
I'd appreciate any help – thank you!
Well, that's the basic functionality of transaction - it trakcs the set of fields you provided so if you want a transaction over "msgid delivery", it tracks both those fields and if one of them changes, you have another transaction. That's what you asked splunk to do 😉
If you want separate deliveries tracked within a single transaction, you probably should track transaction only over msgid field.
Indeed, they don't. That complicates things because in this particular case you can infer from the delivery number and sequence in the overall log that it is from the msgid=1 transaction. But in general - are you sure that those deliveries won't interleave with other ones? From different msgids?