topic Re: How to prevent curly brackets from showing up in JSON field names? in Getting Data In

How to prevent curly brackets from showing up in JSON field names?

jravida — Mon, 13 Jul 2015 18:18:49 GMT

Hi folks,

I'm trying to ingest some JSON data into Splunk, which it handles wonderfully, but I am getting curly brackets in my field names, and this is screwing up some searches. I'm not sure why it is placing these in the field names, as my syntax seems correct:

{
    "vulnerable_products": [
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "8.0.0"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "8.0.1"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "8.0.2"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.0.6"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.0.5"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "6.2.2"
        },
        {
            "product": "apple_tv",
            "part": "a",
            "vendor": "apple",
            "version": "7.0.2"
        },
        {
            "product": "iphone_os",
            "part": "o",
            "vendor": "apple",
            "version": "8.1.2"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.0.2"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.1.1"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.0.1"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.1.0"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.0.4"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.0.3"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.1.2"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.0"
        }
    ],
    "modified": "2015-07-05T21:59:19.410-04:00",
    "summary": "WebKit, as used in Apple iOS before 8.1.3; Apple Safari before 6.2.3, 7.x before 7.1.3, and 8.x before 8.0.3; and Apple TV before 7.0.3, allows remote attackers to execute arbitrary code or cause a denial of service (memory corruption and application crash) via a crafted web site, a different vulnerability than CVE-2014-4476 and CVE-2014-4479.",
    "published": "2015-01-30T06:59:11.047-05:00",
    "id": "CVE-2014-4477",
    "cvss": {
        "generated-on-datetime": "2015-01-31T17:28:55.997-05:00",
        "availability-impact": "PARTIAL",
        "integrity-impact": "PARTIAL",
        "access-complexity": "MEDIUM",
        "source": "http://nvd.nist.gov",
        "authentication": "NONE",
        "score": 6.8,
        "access-vector": "NETWORK",
        "confidentiality-impact": "PARTIAL"
    }
}

The problem is with the arrays contained in vulnerable_products turns into:
vulnerable_products{}.product , and having the curly brackets breaks eval commands, even with ' ' placed around the field name. I'd like to just remove them, but can't figure out how my syntax is wrong...

Is there anything I need to do with this data to eliminate the {} from the field name?

Re: How to prevent curly brackets from showing up in JSON field names?

woodcock — Mon, 13 Jul 2015 19:04:39 GMT

You can refer to the field name literally using $ as $vulnerable_products{}.product$ . Also you can rename it like this:

... | rename *{}* AS **

Re: How to prevent curly brackets from showing up in JSON field names?

jravida — Tue, 29 Sep 2020 06:40:16 GMT

I did try rename, but say if I pass it to a second eval, it shows no result. If I do the same thing with fields that have no {} it will work every time, as soon as I replace the test field with the {} one:

index = json_test | eval products='vulnerable_products{}.product' | eval new_field=products. "," .date_zone

is broken

index = json_test | eval products='date_year' | eval new_field=products. "," .date_zone

works fine.

So when the curly brackets are in a field name, it breaks evals downstream. Must be a bug. I want to reformat the data in whichever way, I don't need to use the test data syntax necessarily.

Re: How to prevent curly brackets from showing up in JSON field names?

woodcock — Mon, 13 Jul 2015 19:22:20 GMT

I just told you how to handle referring to the variable; did you try it?

index = json_test | eval products=$vulnerable_products{}.product$ | eval new_field=products. "," .date_zone

Re: How to prevent curly brackets from showing up in JSON field names?

jravida — Mon, 13 Jul 2015 19:39:01 GMT

Yes that worked, I missed over the $. I was using single quotes. Awesome!

Re: How to prevent curly brackets from showing up in JSON field names?

rbreton — Fri, 11 Aug 2017 13:50:51 GMT

The dollar sign ($) trick is just what I needed.