topic Re: How can I parse a log containing multiple JSON records? in Splunk Search

How can I parse a log containing multiple JSON records?

yk010123 — Tue, 04 Oct 2022 15:10:25 GMT

I have the following log:

Requests over Threshold found: {"kv":{"top_requests":[{"operation_name":"get","last_dispatch_duration_us":136231,"last_remote_socket":"xx","last_local_id":"67B57F7300000001/00000000C1E2DBA3","last_local_socket":"xxx:37894","total_dispatch_duration_us":136231,"total_server_duration_us":3,"operation_id":"0x127f1","timeout_ms":250,"last_server_duration_us":3,"total_duration_us":136516},{"operation_name":"get","last_dispatch_duration_us":135914,"last_remote_socket":"xxx","last_local_id":"67B57F7300000001/00000000C1E2DBA3","last_local_socket":"xxx:37894","total_dispatch_duration_us":135914,"total_server_duration_us":15,"operation_id":"0x127e9","timeout_ms":250,"last_server_duration_us":15,"total_duration_us":135985},{"operation_name":"get","last_dispatch_duration_us":135827,"last_remote_socket":"xxx.xxx:11210","last_local_id":"67B57F7300000001/000000006A92D90B","last_local_socket":"xxx:59306","total_dispatch_duration_us":135827,"total_server_duration_us":15,"operation_id":"0x127e7","timeout_ms":250,"last_server_duration_us":15,"total_duration_us":135946}],"total_count":3}}

How can I parse this?

Re: How can I parse a log containing multiple JSON records?

ITWhisperer — Tue, 04 Oct 2022 16:24:24 GMT

| rex "Requests over Threshold found: (?<json>.*)" | spath input=json

Re: How can I parse a log containing multiple JSON records?

yk010123 — Tue, 04 Oct 2022 16:33:16 GMT

How can I format this to a table?

Is this the right approach?

rex "Requests over Threshold found: (?<json>.*)"| spath input=json | table kv.*

Re: How can I parse a log containing multiple JSON records?

ITWhisperer — Tue, 04 Oct 2022 16:35:12 GMT

It depends on what you are trying to achieve

Re: How can I parse a log containing multiple JSON records?

yk010123 — Tue, 04 Oct 2022 16:48:18 GMT

I am trying to show all the fields in a table format so I can sort and analyze them

Re: How can I parse a log containing multiple JSON records?

ITWhisperer — Tue, 04 Oct 2022 17:04:56 GMT

Depending on what you want - assuming you are just looking at top_requests, you could do something like this

| rex "Requests over Threshold found: (?<json>.*)" | spath input=json kv.top_requests{} output=top_requests | mvexpand top_requests | spath input=top_requests

Re: How can I parse a log containing multiple JSON records?

yk010123 — Tue, 04 Oct 2022 17:07:18 GMT

I would like to show all the fields from the JSON in a table format such that we have field=value

Re: How can I parse a log containing multiple JSON records?

yk010123 — Tue, 04 Oct 2022 17:11:02 GMT

If we have multiple entries in the JSON, it should create individual rows

Re: How can I parse a log containing multiple JSON records?

yk010123 — Tue, 04 Oct 2022 17:13:12 GMT

Perhaps this?

| spath input=json kv.top_requests{} output=top_requests | mvexpand top_requests | spath input=top_requests | table operation_name last_dispatch_duration_us last_remote_socket last_local_id last_local_socket total_dispatch_duration_us total_server_duration_us operation_id timeout_ms last_server_duration_us total_duration_us