How to prevent curly brackets from showing up in JSON field names?

[jravida]

Hi folks,

I'm trying to ingest some JSON data into Splunk, which it handles wonderfully, but I am getting curly brackets in my field names, and this is screwing up some searches. I'm not sure why it is placing these in the field names, as my syntax seems correct:

{
    "vulnerable_products": [
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "8.0.0"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "8.0.1"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "8.0.2"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.0.6"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.0.5"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "6.2.2"
        },
        {
            "product": "apple_tv",
            "part": "a",
            "vendor": "apple",
            "version": "7.0.2"
        },
        {
            "product": "iphone_os",
            "part": "o",
            "vendor": "apple",
            "version": "8.1.2"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.0.2"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.1.1"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.0.1"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.1.0"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.0.4"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.0.3"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.1.2"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.0"
        }
    ],
    "modified": "2015-07-05T21:59:19.410-04:00",
    "summary": "WebKit, as used in Apple iOS before 8.1.3; Apple Safari before 6.2.3, 7.x before 7.1.3, and 8.x before 8.0.3; and Apple TV before 7.0.3, allows remote attackers to execute arbitrary code or cause a denial of service (memory corruption and application crash) via a crafted web site, a different vulnerability than CVE-2014-4476 and CVE-2014-4479.",
    "published": "2015-01-30T06:59:11.047-05:00",
    "id": "CVE-2014-4477",
    "cvss": {
        "generated-on-datetime": "2015-01-31T17:28:55.997-05:00",
        "availability-impact": "PARTIAL",
        "integrity-impact": "PARTIAL",
        "access-complexity": "MEDIUM",
        "source": "http://nvd.nist.gov",
        "authentication": "NONE",
        "score": 6.8,
        "access-vector": "NETWORK",
        "confidentiality-impact": "PARTIAL"
    }
}

The problem is with the arrays contained in vulnerable_products turns into:
vulnerable_products{}.product , and having the curly brackets breaks eval commands, even with ' ' placed around the field name. I'd like to just remove them, but can't figure out how my syntax is wrong...

Is there anything I need to do with this data to eliminate the {} from the field name?

[woodcock · Accepted solution]

You can refer to the field name literally using $ as $vulnerable_products{}.product$. Also you can rename it like this:

... | rename *{}* AS **

[jravida]

I did try rename, but say if I pass it to a second eval, it shows no result. If I do the same thing with fields that have no {} it will work every time, as soon as I replace the test field with the {} one:

index = json_test | eval products='vulnerable_products{}.product' | eval new_field=products. "," .date_zone

is broken

index = json_test | eval products='date_year' | eval new_field=products. "," .date_zone

works fine.

So when the curly brackets are in a field name, it breaks evals downstream. Must be a bug. I want to reformat the data in whichever way, I don't need to use the test data syntax necessarily.

[rbreton]

The dollar sign ($) trick is just what I needed.

[woodcock]

I just told you how to handle referring to the variable; did you try it?

index = json_test | eval products=$vulnerable_products{}.product$ | eval new_field=products. "," .date_zone

[jravida]

Yes that worked, I missed over the $. I was using single quotes. Awesome!