I have events similar to this example:
1) date1, id1, misc
2) date2, id2, misc
3) date3, , misc
4) date4, id3 and id4, misc
The ids in 4) should be split into two separate lines. The result should look like this:
4) date4, id3 , misc
5) date4, id4, misc
But when using makemv with tokenizer lines which do not match, the tokenizers are skipped in the result, e.g.:
... | makemv tokenizer="(id\d)" ID | mvexpand ID | ...
3) date4, id3 , misc
4) date4, id4, misc
How can I keep the non-matching lines? Is there a way to only use makemv where it is necessary?
My workaround at the moment is to append a second search that looks for events with an empty ID and adds those events again after the makemv, e,g,:
first search | ... | makemv tokenizer="(id\d)" ID | mvexpand ID | append [ first search again | ... | search NOT ID="*" | ... ] | ...
But searching twice can't be an optimal solution.
Not sure of the exact format of your data, but look at this example using split/mvexpand where fillnull is used to ensure the missing id gets preserved in the mvexpand
| makeresults count=4
| eval t=1
| accum t
| eval _time=now() - (random() % 86400)
| eval id=case(t=1,"id1",t=2,"id2",t=3,null,t=4,"id3 id4")
| eval misc=random()
| fields - t
| eval id=split(id," ")
| fillnull value="" id
| mvexpand id
It's likely that fillnull will solve your problem in your example makemv case also - set the value to whatever makes sense in your context.
Hope this helps
View solution in original post
great idea - this put me on the right track.
What I do now is to use an eval before the makemv that creates an ID value if empty, that matches my tokenizer but cannot occur in the real data.
After the the mvexpand I use another eval to remove this dummy-ID again.
Seems to work fine. And two evals should be better than two searches.
Thank you very much and kind regards,