topic Re: Parse nested json array without direct key-value mapping in Splunk Search

Parse nested json array without direct key-value mapping

rashmeet — Fri, 17 Jul 2020 00:28:36 GMT

Hi,

I have a json that looks like the following -

{
"id": "123",
"uri": "http://xyz.com/api",
"method": "POST",
"headers": [
"Accept: application/json",
"SERVICE.ENV: qa",
"SERVICE.NAME: someservice",
"CLIENT.ID: s0m3id",
"CLIENT_TYPE: typeA",
"CLIENT_IP:123.456.7.8"
],
"cookies": [],
"message": "Request Finished",
"status": 200
}

Within the headers section, I want to capture what all CLIENT_IPs are passing other header info such as SERVICE.ENV and SERVICE.NAME. The catch being, CLIENT_IP:123.456.7.8 is all in a single pair of quotes, so it isn't being parsed as a key value pair (as per my understanding). Please help.

Re: Parse nested json array without direct key-value mapping

_gkollias — Fri, 17 Jul 2020 02:01:03 GMT

Hello -

This looks like valid JSON. Have you tried setting KV_MODE=json in props.conf?

https://docs.splunk.com/Documentation/Splunk/8.0.5/Knowledge/Automatickey-valuefieldextractionsatsearch-time

Thanks!

Re: Parse nested json array without direct key-value mapping

rashmeet — Fri, 17 Jul 2020 17:12:38 GMT

@_gkollias - Thank you for the response.
And sorry I’m absolutely new to splunk which is why I was unaware for the KV_MODE. So once it’s specified, will I be able to query with the key such as CLIENT_ID?

I've been trying queries like -

index=my_service
| rename @fields.headers{}.* as *
| eval a = mvzip(CLIENT_IP,CLIENT.ID,"|")
| mvexpand a
| table CLIENT_IP,CLIENT.ID

And it is giving me empty table.

Re: Parse nested json array without direct key-value mapping

rashmeet — Fri, 17 Jul 2020 17:12:56 GMT

Re: Parse nested json array without direct key-value mapping

to4kawa — Fri, 17 Jul 2020 18:32:11 GMT

index=_internal |head 1 | fields _time _raw |eval _raw="{\"id\":\"123\",\"uri\":\"http://xyz.com/api\",\"method\":\"POST\",\"headers\":[\"Accept: application/json\",\"SERVICE.ENV: qa\",\"SERVICE.NAME: someservice\",\"CLIENT.ID: s0m3id\",\"CLIENT_TYPE: typeA\",\"CLIENT_IP:123.456.7.8\"],\"cookies\":[],\"message\":\"Request Finished\",\"status\":200}" | spath headers{} output=headers | rex field=headers max_match=0 "(?<key>\w+):\s*(?<value>\S+)" | eval _raw=mvzip(key,value,"=") | kv

This query extracts header array.

Re: Parse nested json array without direct key-value mapping

rashmeet — Sat, 18 Jul 2020 16:48:56 GMT

@to4kawa - Thank you, that does help, much appreciated. 🙂