Given input like this:
id, action, message
1, add, Adding this thing
2, add, Adding this other thing
, , I am a different message
1, destroy, Remove this thing
, , I am yet a different message
2, destroy, Remove this other thing
, , And I am yet a different message
I want to get:
activeids, id, action, message
1, 1, add, Adding this thing
1;2, 2, add, Adding this other thing
1;2, , , I am a different message
2, 1, destroy, Remove this thing
2, , , I am yet a different message
, 2, destroy, Remove this other thing
, , , And I am yet a different message
I've been fighting with streamstats global=false current=false window=1 last(activeids) as activeids and a load of eval statements, but streamstats doesn't seem to calculate the values when I think it should (in order of execution per event).
When does streamstats actually do its work vs. other statements in the pipeline?
Here's a query that illustrates what I'm trying to do:
| stats count
| eval r="message=No_id_yet"
| eval r=mvappend(r,"id=1 action=add message=Adding_this_id")
| eval r=mvappend(r,"id=2 action=add message=Adding_this_other_id")
| eval r=mvappend(r,"message=Im_a_different_message")
| eval r=mvappend(r,"id=1 action=destroy message=Remove_this_thing")
| eval r=mvappend(r,"message=Im_yet_a_different_message")
| eval r=mvappend(r,"id=2 action=destroy message=Remove_this_other_thing")
| eval r=mvappend(r,"message=And_Im_yet_a_different_message")
| mvexpand r | rename r as _raw | extract
| table id action message
| streamstats window=1 current=false last(activeids) as activeids_prev
| fillnull activeids_prev
| eval add=if(action=="add", id, null)
| eval remove=if(action=="destroy", id, null)
| eval activeids=if(isnotnull(add), mvdedup(mvappend(activeids_prev,add)), activeids_prev)
| eval activeids=if(isnotnull(remove),split( replace( mvjoin(activeids, "IMPOSSIBLEDELIMITER") , remove, "") , "IMPOSSIBLEDELIMITER" ), activeids )
What I see in the results is that the last(activeids) as activeids_prev doesn't actually match anything. It seems that the eval statements are happening before the streamstats.
What am I missing?
