Getting Data In

Parsing JSON data

Sriram
Communicator

I need to build a dashboard to parse the json data and show it more like Tree Structure.What is the best way, I can build a data structure to be able to run custom queries. I tries use basic spath command as well as using jsontutils jsonkvrecursive command with limited success. Appreciate any help. Here is a sample json data.

{

"timings": {
    "cat": "ROOT",
    "el": 597086853,
    "cpu": 256429000,
    "subtimings": {
        "find": {
            "cat": "CODE",
            "el": 500467225,
            "cpu": 218036000,
            "subtimings": {
                "Find - buildMapperObjects": {
                    "cat": "CODE",
                    "el": 6585459,
                    "cpu": 3989000
                },
                "findBySubscriber": {
                    "cat": "CODE",
                    "el": 488985754,
                    "cpu": 211558000,
                    "subtimings": {
                        "findResponseObjects": {
                            "cat": "CODE",
                            "el": 515299,
                            "cpu": 328000
                        },
                        "findSubSqlParamSrc": {
                            "cat": "CODE",
                            "el": 1483307,
                            "cpu": 930000
                        },
                        "executeSPCall": {
                            "cat": "CODE",
                            "el": 152860617,
                            "cpu": 36395000,
                            "aggregates": {
                                "dataSourceCall": {
                                    "cnt": 2,
                                    "el": 22312142
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

}

gcoles
Communicator

I'm not sure about the dashboard part of the request, but you'd find it easier if you set KV_MODE=JSON in a props.conf entry referencing your sourcetype (on the indexer/search heads). I like to set the sourcetype=json for the json input, and then I only need to set up one props.conf entry for all data that I'll be ingesting later.

The KV_MODE automatically runs spath-like field extractions over the entire event, so you can run a search like:

sourcetype=json source=xyz timings | stats count by timings.cat

or

sourcetype=json source=xyz timings.cpu>1000000

Also, if you run your search in "smart" or "verbose" mode, you'll see all your contextual fields on the left with statistics (eg. timings.subtimings.find.subtimings.cat). With that in place, it will be easier to glob on fields for charting purposes.

I think your actual data structure is somewhat cumbersome, though. You may want to have an entity like timings.subtimings.operation = find, rather than find being a key underneath subtimings, that way you can run stats over the values of timings.subtimings.operation, which you could do using your structure. The same principle applies further in the nest. Hope that helps a little bit.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...