topic Re: How do you parse the following JSON data? in Monitoring Splunk

How do you parse the following JSON data?

ips_mandar — Mon, 24 Dec 2018 09:30:55 GMT

Hi,

I want to parse below json data .Below is one sample event-
Objabco.codecnullavro.schema�{"type":"record","name":"Eventtable","namespace":"abc.cdf.ghi","fields":[{"name":"SequenceNumber","type":"long"},{"name":"Offset","type":"string"},{"name":"EnqueuedTimeUtc","type":"string"},{"name":"SystemProperties","type":{"type":"map","values":["long","double","string","bytes"]}},{"name":"Properties","type":{"type":"map","values":["long","double","string","bytes","null"]}},{"name":"Body","type":["null","bytes"]}]}O��D��=XTPO��D��=XTP

I tried spath but unable to get success. Can anyone help me to parse this json data

Re: How do you parse the following JSON data?

whrg — Mon, 24 Dec 2018 10:32:35 GMT

Hello @ips_mandar,

Your sample event does not consist of strict JSON data because of the non-JSON prefix and suffix.

I suggest you extract the JSON data as a new field and then run spath on this field:

yourbasesearch
| rex field=_raw "(?<json_data>\{.+\})"
| spath input=json_data

The regex above is defined very broadly. Your sample event is full of strange symbols. So you might want to improve the regular expression.

Ideally, you would index pure JSON data in Splunk and set the sourcetype to json. This way, the JSON data gets parsed automatically.

Re: How do you parse the following JSON data?

ips_mandar — Mon, 24 Dec 2018 11:16:03 GMT

thanks @whrg
it is working at search time..
also to work at indextime if I remove all strange symbols before and after json data then simple spath command also might work.
Can you please help me to remove all strange symbols before indexing so it can parse the json data at indextime

Re: How do you parse the following JSON data?

whrg — Tue, 25 Dec 2018 10:53:02 GMT

Perhaps you could modify the script which produces the log files. Modify the script that it only produces pure JSON without these headers and footers.

If that's not feasible, check out this thread: Remove first part of string before creating a JSON source type.

So I would suggest you put something like this in your props.conf:

[source::/your/logfile]
SEDCMD-remove_header = s/^.*?\{/{/1