Splunk Search

How to efficiently expand json array entries?

Splunk_321
Path Finder

I have a scenario where i want to expand the field and show as individual events.

Below is my query, which works fine for smaller intervals of time, but larger intervals its not efficient.

index=app_pcf AND cf_app_name="myApp" AND message_type=OUT AND msg.logger=c.m.c.d.MatchesApiDelegateImpl
| spath "msg.logMessage.matched_locations{}.locationId"
| search "msg.logMessage.numReturnedMatches">0
| mvexpand "msg.logMessage.matched_locations{}.locationId"
| fields "msg.logMessage.matched_locations{}.locationId"
| rename "msg.logMessage.matched_locations{}.locationId" to LocationId
| table LocationId

I have a json array called matched_locations which has field locationId. I can have atmost 10 locationIds in a matched_locations I have thousands of events in the duration which will have this matched_locations json array.

Below is example of one such event with bunch of matched_locations

###########################################################

cf_app_name: myApp
cf_org_name: myOrg
cf_space_name: mySpace
job: diego_cell
message_type: OUT
msg: {
application: myApp
correlationid: 0.af277368.1669261134.5eb2322
httpmethod: GET
level: INFO
logMessage: {
apiName: Matches
apiStatus: Success
clientId: oh_HSuoA6jKe0b75gjOIL32gtt1NsygFiutBdALv5b45fe4b
error: NA
matched_locations: [
{
city: PHOENIX
countryCode: USA
locationId: bef26c03-dc5d-4f16-a3ff-957beea80482
matchRank: 1
merchantName: BIG D FLOORCOVERING SUPPLIES
postalCode: 85009-1716
state: AZ
streetAddress: 2802 W VIRGINIA AVE
}
{
city: PHOENIX
countryCode: USA
locationId: ec9b385d-6283-46f4-8c9e-dbbe41e48fcc
matchRank: 2
merchantName: BIG D FLOOR COVERING 4
postalCode: 85009
state: AZ
streetAddress: 4110 W WASHINGTON ST STE 100
}
{ [+]
}
{ [+]
}
{ [+]
}
{ [+]
}
{ [+]
}
{ [+]
}
{ [+]
}
{ [+]
}
]
numReturnedMatches: 10
}
logger: c.m.c.d.MatchesApiDelegateImpl

}
origin: rep
source_instance: 1
source_type: APP/PROC/WEB
timestamp: 1669261139716063000
}

###########################################################

Can anyone help me with how I can expand this field efficiently?

Thank you.

Labels (1)
0 Karma

yuanliu
SplunkTrust
SplunkTrust

Can you define "not efficient"? (Also, when you illustrate JSON data, please use conformant JSON format, not Splunk's preformatted form.)

One improvement I can see is to put "msg.logMessage.numReturnedMatches">0 in the main search.  If mvexpand is too expensive, try stats.  For example, what you are trying to tabulate can be achieved by

index=app_pcf AND cf_app_name="myApp" AND message_type=OUT AND msg.logger=c.m.c.d.MatchesApiDelegateImpl 
msg.logMessage.numReturnedMatches > 0
| stats count by "msg.logMessage.matched_locations{}.locationId"
| fields - count
| rename "msg.logMessage.matched_locations{}.locationId" AS LocationId
Tags (1)
0 Karma
Get Updates on the Splunk Community!

The OpenTelemetry Certified Associate (OTCA) Exam

What’s this OTCA exam? The Linux Foundation offers the OpenTelemetry Certified Associate (OTCA) credential to ...

From Manual to Agentic: Level Up Your SOC at Cisco Live

Welcome to the Era of the Agentic SOC   Are you tired of being a manual alert responder? The security ...

Splunk Classroom Chronicles: Training Tales and Testimonials (Episode 4)

Welcome back to Splunk Classroom Chronicles, our ongoing series where we shine a light on what really happens ...