In my example above, I have multi-value fields. I don't see multi-value variables being used in your example. Did I miss something? Your goal is to have fields 'a', 'c', 'arbitraryKey' (which preexists from Splunk) to be presented with the result you wanted. Those multi-valued field 'keys' and 'values' are not useful; in fact, they only get in the way because of how SPL works. Now, the central problem is number of events. Here, you need to clarify the requirement: Do you want to retain order in each field 'a', 'c', 'arbitraryKey', etc.? If not, simply replace list with values. There is no limit to number of values. But each field will be ordered according to their own ASCII order, therefore losing all correlation. The number of values in each field will also likely be different. If order is important, technically you can still get what you wanted. I will demonstrate one technique. But I highly doubt if it's worth it if you already worry about the cost of mvexpand. First the technique. This illustration uses your knowledge about the dozen or so fields of interest. | fillnull value="null"
| eval of_interest = a . ":" . c . ":" . arbitraryKey . ":" . other1 . ":" . other2 ``` and so on ```
| eval interest_order = mvappend("a", "c", "arbitraryKey", "other1", "other2" ``` and so on ```)
| eventstats values(of_interest) as of_interest
| foreach a c arbitraryKey other1 other2 ``` and so on ```
[eval <<FIELD>> = mvmap(of_interest, mvindex(split(of_interest, mvfind(interest_order, "<<FIELD>>")))) ] Here, the mvmap and split are repeated n times in each row. Is this really less expensive than mvexpand? Maybe you should think through the actual "interesting statistics" and decide if adding a dozen fields with thousands of values to each of thousands of events is truly that useful. And even if it is, maybe of_interest as a whole is better saved into a lookup file instead of as multivalue fields in events.
... View more