Getting Data In

How to parse Json and extract all fields

philschneiderax
New Member

Hello,
I have a logstatement that contains a json.
I am able to parse the json as field. I am also able to parse each field of the json. But only each field by hand.
Is there anyway of like parsing all 1st level fields by hand?

My result is this:

20191119:132817.646  64281752e393 [EJB default - 7] WARN  com.company.MyClass - My Textwarning  –  ID 1,111,111,111 ID2 12313. DataJson: {  "antragKopf": "kopfdata",  "gesperrt": false,  "roid": "12321",  "rolle": "TEST",  "sprache": "de",  "letzterVersuch": "2019-11-19 13:28:02.876",  "anzahlVersuche": 12,  "aktStatus": "MY_STATUS",  "fehlerInfo": "myerror",  "erstelltAm": "2019-11-18 16:18:35.244",  "policennummer": 123,  "version": 12,  "systemmeldung": null,  "isNotwendig": true,  "voData": null,  "abbruchGrund": "NPNE",  "id": 11111}

I can get the JSON data with this query

app=opc javaclass="com.company.MyClass" "My Textwarning" | rex "DataJson: (?<datajson>.*)"

Now I want to extract all first level json attributes as fields (I want to use the fields in an email later)

app=opc javaclass="com.company.MyClass" "My Textwarning" | rex "DataJson: (?<datajson>.*)"  | eval json-antragKopf = spath(datajson, "antragKopf") | eval json-gesperrt = spath(datajson, "gesperrt") | eval json-gesperrt = spath(datajson, "gesperrt") | 

Is there any simple way of extracting all attributes without specifying each one by hand?

0 Karma

dmarling
Builder

You can use this command on the datajson field you extracted to grab all fields: | spath input=datajson
Here's a run anywhere example using your data:

| makeresults count=1
| eval data=" 20191119:132817.646  64281752e393 [EJB default - 7] WARN  com.company.MyClass - My Textwarning  –  ID 1,111,111,111 ID2 12313. DataJson: {  \"antragKopf\": \"kopfdata\",  \"gesperrt\": false,  \"roid\": \"12321\",  \"rolle\": \"TEST\",  \"sprache\": \"de\",  \"letzterVersuch\": \"2019-11-19 13:28:02.876\",  \"anzahlVersuche\": 12,  \"aktStatus\": \"MY_STATUS\",  \"fehlerInfo\": \"myerror\",  \"erstelltAm\": \"2019-11-18 16:18:35.244\",  \"policennummer\": 123,  \"version\": 12,  \"systemmeldung\": null,  \"isNotwendig\": true,  \"voData\": null,  \"abbruchGrund\": \"NPNE\",  \"id\": 11111}"
| rex field=data "DataJson: (?<datajson>.*)"
| spath input=datajson

Be careful with this though as this can use a ton of memory and disk on a search if you have a large amount of data. It's better to identify what fields you need from the json and do either rex or eval spath the fields out like you have been, but if that is not working for you the above method will accomplish what you are asking for.

If this comment/answer was helpful, please up vote it. Thank you.
Get Updates on the Splunk Community!

Operationalizing TDIR: Building a More Resilient, Scalable SOC

Optimizing SOC workflows with a unified, risk-based approach to Threat Detection, Investigation, and Response ...

Almost Too Eventful Assurance: Part 1

Modern IT and Network teams still struggle with too many alerts and isolating issues before they are notified. ...

Demo Day: Strengthen Your SOC with Splunk Enterprise Security 8.1

Today’s threat landscape is more complex than ever. Security operation centers (SOCs) are overwhelmed with ...