I have a scenario where i want to expand the field and show as individual events.
Below is my query, which works fine for smaller intervals of time, but larger intervals its not efficient.
index=app_pcf AND cf_app_name="myApp" AND message_type=OUT AND msg.logger=c.m.c.d.MatchesApiDelegateImpl
| spath "msg.logMessage.matched_locations{}.locationId"
| search "msg.logMessage.numReturnedMatches">0
| mvexpand "msg.logMessage.matched_locations{}.locationId"
| fields "msg.logMessage.matched_locations{}.locationId"
| rename "msg.logMessage.matched_locations{}.locationId" to LocationId
| table LocationId
I have a json array called matched_locations which has field locationId. I can have atmost 10 locationIds in a matched_locations I have thousands of events in the duration which will have this matched_locations json array.
Below is example of one such event with bunch of matched_locations
###########################################################
cf_app_name: myApp
cf_org_name: myOrg
cf_space_name: mySpace
job: diego_cell
message_type: OUT
msg: {
application: myApp
correlationid: 0.af277368.1669261134.5eb2322
httpmethod: GET
level: INFO
logMessage: {
apiName: Matches
apiStatus: Success
clientId: oh_HSuoA6jKe0b75gjOIL32gtt1NsygFiutBdALv5b45fe4b
error: NA
matched_locations: [
{
city: PHOENIX
countryCode: USA
locationId: bef26c03-dc5d-4f16-a3ff-957beea80482
matchRank: 1
merchantName: BIG D FLOORCOVERING SUPPLIES
postalCode: 85009-1716
state: AZ
streetAddress: 2802 W VIRGINIA AVE
}
{
city: PHOENIX
countryCode: USA
locationId: ec9b385d-6283-46f4-8c9e-dbbe41e48fcc
matchRank: 2
merchantName: BIG D FLOOR COVERING 4
postalCode: 85009
state: AZ
streetAddress: 4110 W WASHINGTON ST STE 100
}
{ [+]
}
{ [+]
}
{ [+]
}
{ [+]
}
{ [+]
}
{ [+]
}
{ [+]
}
{ [+]
}
]
numReturnedMatches: 10
}
logger: c.m.c.d.MatchesApiDelegateImpl
}
origin: rep
source_instance: 1
source_type: APP/PROC/WEB
timestamp: 1669261139716063000
}
###########################################################
Can anyone help me with how I can expand this field efficiently?
Thank you.
Can you define "not efficient"? (Also, when you illustrate JSON data, please use conformant JSON format, not Splunk's preformatted form.)
One improvement I can see is to put "msg.logMessage.numReturnedMatches">0 in the main search. If mvexpand is too expensive, try stats. For example, what you are trying to tabulate can be achieved by
index=app_pcf AND cf_app_name="myApp" AND message_type=OUT AND msg.logger=c.m.c.d.MatchesApiDelegateImpl
msg.logMessage.numReturnedMatches > 0
| stats count by "msg.logMessage.matched_locations{}.locationId"
| fields - count
| rename "msg.logMessage.matched_locations{}.locationId" AS LocationId