Alternatives to mvexpand mvzip to create a summary index

[msyparker]

Greetings,

I have a JSON with the format:

bigfield:   [   [-] 
        {   [-] 
         field1:     xxxx
         field2:     true   
         otherfields: wwww
        }   
        {   [-] 
         field1: yyyyy
         field2:     false  
         otherfields:    zzzz
        }   
    ]

and I need to create a summary index to give me the following:
field1 field2 time
xxxx true time
yyyy false time
(xxx must be with true and yyy must be with false)

I'm currently using:

| fields bigfield.field1 bigfield.field2
| foreach * 
    [ eval field1=if('bigfield.field1'!="",'bigfield.field1', "NA"), field2=if('bigfield.field2'!="",'bigfield.field2', "NA")]
| field1 field2 
| eval zipped=mvzip(field1, field2, ";;")
| mvexpand zipped 
| eval zipped = split(zipped, ";;") 
| foreach * 
    [ eval field1 = mvindex(zipped, 0), field2=mvindex(zipped, 1)]
| bin span=1m  _time 
| stats count as "Total" by _time field1 field2

but mvzip and mvexpand consume too much and I get the results truncated:
"[server] command.mvexpand: output will be truncated at ##### results due to excessive memory usage. "

I can't change the threshold, so I was hoping there was a way to make the search less consuming.

[kamlesh_vaghela · Accepted solution]

@msyparker

I have tried with your given sample event.

{
    "bigfield": [{
        "field1": "xxxx",
        "field2": "true",
        "otherfields": "wwww"
    }, {
        "field1": "yyyyy",
        "field2": "false",
        "otherfields": "zzzz"
    }]
}

Can you please try below the search?

YOUR_SEARCH | eval temp=mvzip(bigfield_field1,bigfield_field2) | stats count by _time temp | eval field1=mvindex(split(temp,","),0),field2=mvindex(split(temp,","),1) | table _time field1 field2

My Sample Search

| makeresults 
| eval _raw="{\"bigfield\": [ { \"field1\":\"xxxx\",\"field2\":\"true\",\"otherfields\":\"wwww\"},{ \"field1\":\"yyyyy\",\"field2\":\"false\",\"otherfields\":\"zzzz\"}]}" | kv | rename bigfield{}.* as bigfield_* | eval temp=mvzip(bigfield_field1,bigfield_field2) | stats count by _time temp | eval field1=mvindex(split(temp,","),0),field2=mvindex(split(temp,","),1) | table _time field1 field2

Thanks

[msyparker]

Thank you for your reply!