topic Re: How to extract key value pairs from JSON with a variable key through HTTP Event Collector? in Getting Data In

How to extract key value pairs from JSON with a variable key through HTTP Event Collector?

cloudshowbob — Sun, 07 Jun 2020 01:06:50 GMT

I need help with the following JSON format which is coming from HTTP Event Collector. I want to extract Status, Severity, Id and PatchState from the following JSON format:

{
    "relatedEvents": [],
    "relationships": [
        {
            "resourceId": "REDACTED
            "resourceType": "AWS::SSM::ManagedInstanceInventory",
            "name": "Is associated with "
        }
    ],
    "configuration": {
        "AWS:ComplianceItem": {
            "SchemaVersion": "1.0",
            "Content": {
                "Patch": {
                    "SomeValue": {
                        "Status": "NON_COMPLIANT",
                        "InstalledTime": "",
                        "ExecutionType": "Command",
                        "PatchSeverity": "",
                        "Title": "AAAAAAAA",
                        "Severity": "UNSPECIFIED",
                        "ComplianceType": "Patch",
                        "Classification": "",
                        "DocumentVersion": "",
                        "Id": "BBBBB",
                        "PatchState": "Missing",
                        "PatchBaselineId": "pb-xxxxxxxxxxxxxxxx",
                        "DocumentName": "",
                        "PatchGroup": ""
                    },
                    "SomeOtherValue": {
                        "Status": "NON_COMPLIANT",
                        "InstalledTime": "",
                        "ExecutionType": "Command",
                        "PatchSeverity": "",
                        "Title": "CCCCCCCC",
                        "Severity": "UNSPECIFIED",
                        "ComplianceType": "Patch",
                        "Classification": "",
                        "DocumentVersion": "",
                        "Id": "AAAAAAA",
                        "PatchState": "Missing",
                        "PatchBaselineId": "pb-xxxxxxx",
                        "DocumentName": "",
                        "PatchGroup": ""
                    },

Please note that the embedded nesting's 4th element is a variable (usually a package name) so it is hard to parse using spath and I do not have a fixed number of the 4th nested JSON objects I receive.

Please help and thanks in advance.

Re: How to extract key value pairs from JSON with a variable key through HTTP Event Collector?

to4kawa — Wed, 20 May 2020 02:46:37 GMT

maybe, your log is one line
use Show as raw text and provide them.

and in your json-like log , "Patch": is array [ , isn't it?

Re: How to extract key value pairs from JSON with a variable key through HTTP Event Collector?

cloudshowbob — Wed, 20 May 2020 17:26:34 GMT

I am giving a subset, the raw json is like 10k+ lines. There are no arrays just embedded json objects

Re: How to extract key value pairs from JSON with a variable key through HTTP Event Collector?

to4kawa — Wed, 20 May 2020 21:04:02 GMT

there is the array relationships in your sample.

good luck.

Re: How to extract key value pairs from JSON with a variable key through HTTP Event Collector?

ruman_splunk — Thu, 28 May 2020 21:00:07 GMT

Do an EXTRACT in props.conf that completely ignores the fact that it's JSON 🙂 e.g.

[foo]
EXTRACT-PatchState = "PatchState: \"(?<PatchState>[^\"]+)\","