topic Re: Exracting a json field data from a log in Getting Data In

Exracting a json field data from a log

soumyacharya91 — Tue, 29 Sep 2020 19:04:32 GMT

Hi All,

I am trying to extact a JSON field from the log. I can able to get the data by using "spath input" command. Can you help me with this command's configuration in the back end file so that it can automatically do the job.

search query :

index=*|spath input=<JSON field name>

I need to update this search in props.conf / transforms.conf.

Please find more details about it from the sample log below as I can see it's bit confusing.

"Master_field": "value", "Master_field": "{\"field1\":\"***\",\"field2\":\"value2\",\"field3\":\"value3\",\"field4\":\"value4\",\"field5\":\"{value5}\",\"field6\":\"value6\"},"Master_field": "value"}

Here is the raw data I'm getting in the events. Master fields have been parsed successfully. But the sub fields are not able to parse and they are coming under respective master field.

Using spath command in the query I am able to get the exact data but how to implement the same in the back end file to avoid using this command at search time.

Thanks,
Sam

Re: Exracting a json field data from a log

p_gurav — Thu, 19 Apr 2018 10:49:30 GMT

Did you try putting KV_MODE=_json OR INDEXED_EXTRACTION = _json?

Re: Exracting a json field data from a log

soumyacharya91 — Thu, 19 Apr 2018 10:52:20 GMT

Yes, already tried KV_MODE=JSON but no luck 😞

Re: Exracting a json field data from a log

ravidudala — Tue, 29 Sep 2020 19:04:34 GMT

You can try the following in props.conf

[sourcteype]
INDEXED_EXTRACTIONS = json
KV_MODE = none

Re: Exracting a json field data from a log

jlvix1 — Tue, 29 Sep 2020 19:08:50 GMT

I think if you have the data in splunk as sourcetype _json or json_no_timestamp you should be ok, see how this turns up in a verbose mode search and there may not be a need to use INDEXED_EXTRACTIONS=json or KV_MODE = none or spath queries.

There are issues when you have data that "contains" JSON, e.g. an exception message etc... That's when problems begin, is this the case.

Re: Exracting a json field data from a log

soumyacharya91 — Thu, 19 Apr 2018 13:19:18 GMT

I tried it but still getting the same result.

Re: Exracting a json field data from a log

ddrillic — Thu, 19 Apr 2018 13:20:14 GMT

We had the following thread about this subject at - What are the requirements for a perfect Splunk JSON document?

The following worked for us -

INDEXED_EXTRACTIONS = json
category = Structured

Re: Exracting a json field data from a log

soumyacharya91 — Fri, 20 Apr 2018 14:51:24 GMT

Yes, you are absolutely right we can use this to parse and index the JSON data.