group keys having wildcard char like usermetadata_* by other unique field

[neha19oct97]

Hi All,

I have a requirement to group keys  (key - value pair) having wildcard char like - usermetadata_*  by other unique field PipelineName.

INPUT :

level: INFO
   logGroup: test
   loggerName: Logger
   message: {

     Trace-Type: client
     UserMetaData_eventID: [1234]
     UserMetaData_orderLineType: xyz
     UserMetaData_purchaseOrderType: [2]
     UserMetaData_purchaseOrderID: [3421]
     UserMetaData_purchaseOrderVersion: [789]
     UserMetaData_salesOrderID: [-789]
     PipelineName: abc

}

OUTPUT example: 

I want the output like this :

PipelineName                        usermetadata_keys

abc                                            UserMetaData_eventID:
                                                   UserMetaData_orderLineType
                                                   UserMetaData_purchaseOrderType
                                                   UserMetaData_purchaseOrderID
                                                   UserMetaData_purchaseOrderVersion
                                                   UserMetaData_salesOrderID

[to4kawa]

Why do you have many `message` fields?

your JSON log is only one message key. 

[to4kawa]

index=_internal | head 1 |fields _raw | eval _raw="{\"level\":\"INFO\",\"logGroup\":\"test\",\"loggerName\":\"Logger\",\"message\":{\"Trace-Type\":\"client\",\"UserMetaData_eventID\":[1234],\"UserMetaData_orderLineType\":\"xyz\",\"UserMetaData_purchaseOrderType\":[2],\"UserMetaData_purchaseOrderID\":[3421],\"UserMetaData_purchaseOrderVersion\":[789],\"UserMetaData_salesOrderID\":[-789],\"PipelineName\":\"abc\"}}"
| rename COMMENT as "the logic"
| spath message output=message
| spath message.PipelineName output=PipelineName
| rex field=message max_match=0 "\"(?<key>\w+)\":"
| stats values(eval(mvfilter(match(key,"UserMetaData")))) as usermetadata_keys by PipelineName

[neha19oct97]

Hi @to4kawa,

It works if we have only one "message field". If i removed "head 1" from the query it didn't work. Kindly suggest if we have more than one message fields.

Thanks in advance.