Parsing JSON data

Sriram — Fri, 15 Mar 2013 22:37:31 GMT

I need to build a dashboard to parse the json data and show it more like Tree Structure.What is the best way, I can build a data structure to be able to run custom queries. I tries use basic spath command as well as using jsontutils jsonkvrecursive command with limited success. Appreciate any help. Here is a sample json data.

{

"timings": {
    "cat": "ROOT",
    "el": 597086853,
    "cpu": 256429000,
    "subtimings": {
        "find": {
            "cat": "CODE",
            "el": 500467225,
            "cpu": 218036000,
            "subtimings": {
                "Find - buildMapperObjects": {
                    "cat": "CODE",
                    "el": 6585459,
                    "cpu": 3989000
                },
                "findBySubscriber": {
                    "cat": "CODE",
                    "el": 488985754,
                    "cpu": 211558000,
                    "subtimings": {
                        "findResponseObjects": {
                            "cat": "CODE",
                            "el": 515299,
                            "cpu": 328000
                        },
                        "findSubSqlParamSrc": {
                            "cat": "CODE",
                            "el": 1483307,
                            "cpu": 930000
                        },
                        "executeSPCall": {
                            "cat": "CODE",
                            "el": 152860617,
                            "cpu": 36395000,
                            "aggregates": {
                                "dataSourceCall": {
                                    "cnt": 2,
                                    "el": 22312142
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

}

Re: Parsing JSON data

gcoles — Sat, 16 Mar 2013 01:08:51 GMT

I'm not sure about the dashboard part of the request, but you'd find it easier if you set KV_MODE=JSON in a props.conf entry referencing your sourcetype (on the indexer/search heads). I like to set the sourcetype=json for the json input, and then I only need to set up one props.conf entry for all data that I'll be ingesting later.

The KV_MODE automatically runs spath-like field extractions over the entire event, so you can run a search like:

sourcetype=json source=xyz timings | stats count by timings.cat

sourcetype=json source=xyz timings.cpu>1000000

Also, if you run your search in "smart" or "verbose" mode, you'll see all your contextual fields on the left with statistics (eg. timings.subtimings.find.subtimings.cat). With that in place, it will be easier to glob on fields for charting purposes.

I think your actual data structure is somewhat cumbersome, though. You may want to have an entity like timings.subtimings.operation = find, rather than find being a key underneath subtimings, that way you can run stats over the values of timings.subtimings.operation, which you could do using your structure. The same principle applies further in the nest. Hope that helps a little bit.

topic Re: Parsing JSON data in Getting Data In

Parsing JSON data

Re: Parsing JSON data