Getting Data In

How to prevent curly brackets from showing up in JSON field names?

jravida
Communicator

Hi folks,

I'm trying to ingest some JSON data into Splunk, which it handles wonderfully, but I am getting curly brackets in my field names, and this is screwing up some searches. I'm not sure why it is placing these in the field names, as my syntax seems correct:

{
    "vulnerable_products": [
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "8.0.0"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "8.0.1"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "8.0.2"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.0.6"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.0.5"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "6.2.2"
        },
        {
            "product": "apple_tv",
            "part": "a",
            "vendor": "apple",
            "version": "7.0.2"
        },
        {
            "product": "iphone_os",
            "part": "o",
            "vendor": "apple",
            "version": "8.1.2"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.0.2"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.1.1"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.0.1"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.1.0"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.0.4"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.0.3"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.1.2"
        },
        {
            "product": "safari",
            "part": "a",
            "vendor": "apple",
            "version": "7.0"
        }
    ],
    "modified": "2015-07-05T21:59:19.410-04:00",
    "summary": "WebKit, as used in Apple iOS before 8.1.3; Apple Safari before 6.2.3, 7.x before 7.1.3, and 8.x before 8.0.3; and Apple TV before 7.0.3, allows remote attackers to execute arbitrary code or cause a denial of service (memory corruption and application crash) via a crafted web site, a different vulnerability than CVE-2014-4476 and CVE-2014-4479.",
    "published": "2015-01-30T06:59:11.047-05:00",
    "id": "CVE-2014-4477",
    "cvss": {
        "generated-on-datetime": "2015-01-31T17:28:55.997-05:00",
        "availability-impact": "PARTIAL",
        "integrity-impact": "PARTIAL",
        "access-complexity": "MEDIUM",
        "source": "http://nvd.nist.gov",
        "authentication": "NONE",
        "score": 6.8,
        "access-vector": "NETWORK",
        "confidentiality-impact": "PARTIAL"
    }
}

The problem is with the arrays contained in vulnerable_products turns into:
vulnerable_products{}.product , and having the curly brackets breaks eval commands, even with ' ' placed around the field name. I'd like to just remove them, but can't figure out how my syntax is wrong...

Is there anything I need to do with this data to eliminate the {} from the field name?

Tags (2)
0 Karma
1 Solution

woodcock
Esteemed Legend

You can refer to the field name literally using $ as $vulnerable_products{}.product$. Also you can rename it like this:

... | rename *{}* AS **

View solution in original post

woodcock
Esteemed Legend

You can refer to the field name literally using $ as $vulnerable_products{}.product$. Also you can rename it like this:

... | rename *{}* AS **

jravida
Communicator

I did try rename, but say if I pass it to a second eval, it shows no result. If I do the same thing with fields that have no {} it will work every time, as soon as I replace the test field with the {} one:

index = json_test | eval products='vulnerable_products{}.product' | eval new_field=products. "," .date_zone

is broken

index = json_test | eval products='date_year' | eval new_field=products. "," .date_zone

works fine.

So when the curly brackets are in a field name, it breaks evals downstream. Must be a bug. I want to reformat the data in whichever way, I don't need to use the test data syntax necessarily.

0 Karma

rbreton
Path Finder

The dollar sign ($) trick is just what I needed.

0 Karma

woodcock
Esteemed Legend

I just told you how to handle referring to the variable; did you try it?

index = json_test | eval products=$vulnerable_products{}.product$ | eval new_field=products. "," .date_zone
0 Karma

jravida
Communicator

Yes that worked, I missed over the $. I was using single quotes. Awesome!

0 Karma
Get Updates on the Splunk Community!

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...