How can I parse a large JSON block with nested structures and get unique events from it?

a238574 — Tue, 06 Feb 2018 15:03:31 GMT

Need to be able to pull/group and dedup a number of nested JSON events where each record has a single account identifier with multiple resource identifiers. I am new to splunk and need help creating the query to be able to list the unique resource identifiers for each account Identifier. There can be multiple records for each Account identifier and multiple nested resource identifiers that may not be unique. Sample event below. I know JSON syntax is not correct just trying to give an idea of the structure

"Message": {
                    "account_id": "423874236234",
                                         "resources": [{
                        "ResourseId": "ID1",
                    },
                    {
                        "ResourseId": "ID1",
                    },
                    {
                        "ResourseId": "ID2",
                    }]

For each JSON event, the AccountID will be singular but you can have multiple events for each Account ID.
Each event will have multiple ResourceID's and the ResourceIDs can be duplicated.

One more wrinkle. These are large JSON events and I had to use the path variable just to get a complete list of resource ID's out of a single event

This is the query I wrote to get a list of unique resourceIDs for all events:

Index= index1 source="Stuff"    | spath output=ResourceID path=Message.resources{}.ResourceID| stats count by ResourceID

*Names have been changed to protect the innocent * 🙂

Re: How can I parse a large JSON block with nested structures and get unique events from it?

valiquet — Fri, 09 Mar 2018 11:30:28 GMT

If you ingestion is properly configured (props) all variable will be extracted automatically.

topic Re: How can I parse a large JSON block with nested structures and get unique events from it? in Getting Data In

How can I parse a large JSON block with nested structures and get unique events from it?

Re: How can I parse a large JSON block with nested structures and get unique events from it?