Splunk Search

How to efficiently expand json array entries?

Splunk_321
Path Finder

I have a scenario where i want to expand the field and show as individual events.

Below is my query, which works fine for smaller intervals of time, but larger intervals its not efficient.

index=app_pcf AND cf_app_name="myApp" AND message_type=OUT AND msg.logger=c.m.c.d.MatchesApiDelegateImpl
| spath "msg.logMessage.matched_locations{}.locationId"
| search "msg.logMessage.numReturnedMatches">0
| mvexpand "msg.logMessage.matched_locations{}.locationId"
| fields "msg.logMessage.matched_locations{}.locationId"
| rename "msg.logMessage.matched_locations{}.locationId" to LocationId
| table LocationId

I have a json array called matched_locations which has field locationId. I can have atmost 10 locationIds in a matched_locations I have thousands of events in the duration which will have this matched_locations json array.

Below is example of one such event with bunch of matched_locations

###########################################################

cf_app_name: myApp
cf_org_name: myOrg
cf_space_name: mySpace
job: diego_cell
message_type: OUT
msg: {
application: myApp
correlationid: 0.af277368.1669261134.5eb2322
httpmethod: GET
level: INFO
logMessage: {
apiName: Matches
apiStatus: Success
clientId: oh_HSuoA6jKe0b75gjOIL32gtt1NsygFiutBdALv5b45fe4b
error: NA
matched_locations: [
{
city: PHOENIX
countryCode: USA
locationId: bef26c03-dc5d-4f16-a3ff-957beea80482
matchRank: 1
merchantName: BIG D FLOORCOVERING SUPPLIES
postalCode: 85009-1716
state: AZ
streetAddress: 2802 W VIRGINIA AVE
}
{
city: PHOENIX
countryCode: USA
locationId: ec9b385d-6283-46f4-8c9e-dbbe41e48fcc
matchRank: 2
merchantName: BIG D FLOOR COVERING 4
postalCode: 85009
state: AZ
streetAddress: 4110 W WASHINGTON ST STE 100
}
{ [+]
}
{ [+]
}
{ [+]
}
{ [+]
}
{ [+]
}
{ [+]
}
{ [+]
}
{ [+]
}
]
numReturnedMatches: 10
}
logger: c.m.c.d.MatchesApiDelegateImpl

}
origin: rep
source_instance: 1
source_type: APP/PROC/WEB
timestamp: 1669261139716063000
}

###########################################################

Can anyone help me with how I can expand this field efficiently?

Thank you.

Labels (1)
0 Karma

yuanliu
SplunkTrust
SplunkTrust

Can you define "not efficient"? (Also, when you illustrate JSON data, please use conformant JSON format, not Splunk's preformatted form.)

One improvement I can see is to put "msg.logMessage.numReturnedMatches">0 in the main search.  If mvexpand is too expensive, try stats.  For example, what you are trying to tabulate can be achieved by

index=app_pcf AND cf_app_name="myApp" AND message_type=OUT AND msg.logger=c.m.c.d.MatchesApiDelegateImpl 
msg.logMessage.numReturnedMatches > 0
| stats count by "msg.logMessage.matched_locations{}.locationId"
| fields - count
| rename "msg.logMessage.matched_locations{}.locationId" AS LocationId
Tags (1)
0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

Tech Talk Recap | Mastering Threat Hunting

Mastering Threat HuntingDive into the world of threat hunting, exploring the key differences between ...

Observability for AI Applications: Troubleshooting Latency

If you’re working with proprietary company data, you’re probably going to have a locally hosted LLM or many ...

Splunk AI Assistant for SPL vs. ChatGPT: Which One is Better?

In the age of AI, every tool promises to make our lives easier. From summarizing content to writing code, ...