topic Re: How to parse string JSON along with actual JSON? in Splunk Search

How to parse string JSON along with actual JSON?

JChris_ — Thu, 21 Apr 2022 17:52:14 GMT

I have the following log in Splunk:

{ "tags":{ "app":"foobar", "ou":"internal" }, "log":"{\"key1\":\"value1\",\"key2\":\"value2\",\"key3\":\"value3\"}", "timestamp":"2022-04-21T17:00:00.000Z" }

I know I can parse the string JSON into actual JSON and replace the _raw like this:

index=my_index_name | eval _raw=log

But, if I use the SPL above, the timestamp and tags keys would be deleted from the _raw, that's not what I want.

I want to use SPL to parse it in a way where the _raw equals to:

{ "tags":{ "app":"foobar", "ou":"internal" }, "log": { "key1": "value1", "key2": "value2", "key3": "value3" }, "timestamp":"2022-04-21T17:00:00.000Z" }

Re: How to parse string JSON along with actual JSON?

PickleRick — Thu, 21 Apr 2022 18:09:51 GMT

Ok. So you have a json-formatted value inside your json event.

You can approach it from two different angles.

1) Explicitly use spath on that value.

<your_search>
| spath input=log

And I think it's the easiest solution.

2) "Rearrange" your event a bit - remember the old value of _raw, replace it, let Splunk parse it and then restore old _raw. A bit confusing, I know 😉

<your_search>
| eval oldraw=_raw
| eval _raw=log
| extract
| eval _raw=oldraw
| fields - oldraw

Haven't tried it but should work (but it's not a pretty solution).

Re: How to parse string JSON along with actual JSON?

JChris_ — Thu, 21 Apr 2022 18:29:03 GMT

I thought of using spath, it does work but does not override the _raw. I wanted the _raw to also be a complete JSON.

Re: How to parse string JSON along with actual JSON?

PickleRick — Thu, 21 Apr 2022 18:43:34 GMT

Then you'd have to manipulate the event manually. Which could probably be possible in some specific cases but it'd be hard to do in general case. Firstly you'd have to remove the original contents of the field, which is not simply "anything between the quotes" because you'd have to account for escaped quotes (possibly on many levels of nested strings).

But the more importantly - you'd have to reconstruct the json structure. While you have some json_*() functions, you have to know the structure beforehand to create/modify a json object. So if you can have anything in your "log" field... well, I don't see it working.

Re: How to parse string JSON along with actual JSON?

ITWhisperer — Fri, 22 Apr 2022 06:27:34 GMT

Try something like this

| eval _raw=replace(_raw,"\\\\","") | eval _raw=replace(_raw,"\"{","{") | eval _raw=replace(_raw,"}\"","}") | spath