Getting Data In

How to extract key value pairs from JSON with a variable key through HTTP Event Collector?

cloudshowbob
New Member

I need help with the following JSON format which is coming from HTTP Event Collector. I want to extract Status, Severity, Id and PatchState from the following JSON format:

{
    "relatedEvents": [],
    "relationships": [
        {
            "resourceId": "REDACTED
            "resourceType": "AWS::SSM::ManagedInstanceInventory",
            "name": "Is associated with "
        }
    ],
    "configuration": {
        "AWS:ComplianceItem": {
            "SchemaVersion": "1.0",
            "Content": {
                "Patch": {
                    "SomeValue": {
                        "Status": "NON_COMPLIANT",
                        "InstalledTime": "",
                        "ExecutionType": "Command",
                        "PatchSeverity": "",
                        "Title": "AAAAAAAA",
                        "Severity": "UNSPECIFIED",
                        "ComplianceType": "Patch",
                        "Classification": "",
                        "DocumentVersion": "",
                        "Id": "BBBBB",
                        "PatchState": "Missing",
                        "PatchBaselineId": "pb-xxxxxxxxxxxxxxxx",
                        "DocumentName": "",
                        "PatchGroup": ""
                    },
                    "SomeOtherValue": {
                        "Status": "NON_COMPLIANT",
                        "InstalledTime": "",
                        "ExecutionType": "Command",
                        "PatchSeverity": "",
                        "Title": "CCCCCCCC",
                        "Severity": "UNSPECIFIED",
                        "ComplianceType": "Patch",
                        "Classification": "",
                        "DocumentVersion": "",
                        "Id": "AAAAAAA",
                        "PatchState": "Missing",
                        "PatchBaselineId": "pb-xxxxxxx",
                        "DocumentName": "",
                        "PatchGroup": ""
                    },

Please note that the embedded nesting's 4th element is a variable (usually a package name) so it is hard to parse using spath and I do not have a fixed number of the 4th nested JSON objects I receive.

Please help and thanks in advance.

Labels (2)
0 Karma

ruman_splunk
Splunk Employee
Splunk Employee

Do an EXTRACT in props.conf that completely ignores the fact that it's JSON 🙂 e.g.

[foo]
EXTRACT-PatchState = "PatchState: \"(?<PatchState>[^\"]+)\","
0 Karma

to4kawa
Ultra Champion

maybe, your log is one line
use Show as raw text and provide them.

and in your json-like log , "Patch": is array [ , isn't it?

0 Karma

cloudshowbob
New Member

I am giving a subset, the raw json is like 10k+ lines. There are no arrays just embedded json objects

0 Karma

to4kawa
Ultra Champion

there is the array relationships in your sample.

good luck.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Kubernetes Alerts That Work Out of the Box with AutoDetect detectors in Splunk ...

A deployment rolls out slowly, one pod sits not ready for a few minutes, and whoever is on call gets an alert ...

Observability at Your Fingertips Directly From Splunk Platform—for Free

At .conf, we announced our roadmap to bring observability foundations to Splunk Platform. Bringing metrics and ...