Splunk Dev

How to get the depth of one of nested json objects in mixed log line?

Digister
Explorer

Hi guys,

I am stuck on this for hours now, but I can't achieve what I am looking for.
My log lines are looking like this

[2018-08-03T05:14:14.731] [INFO] komvos.controller.scenario - Summary: {"id":"myID","status":"END","scenario":"My Scenario Name","parameters":{"baseUrl":"https://myUrl.com","username":"JohnSmith","isTest":false},"durationInSeconds":"11.614","manualEffortInSeconds":1200,"isCounting":false,"isTest":true,"isStatusEnd":true,"existSuccessfulRunTodayWithSameParameters":false, "result":{"statusCode":"200","type":"myType","description":"myDescription" }

What I want is:

Time     | ID | Scenario | Parameters | Duration in Seconds | Manual Effort in Seconds | isCounting | ... | result

 2018... | myID |My Scenario Name  |  {"baseUrl":"https://myUrl.com","username":"JohnSmith","isTest":false} | 11.614 | 1200 | ... | {"statusCode":"200","type":"myType","description":"myDescription" }

What I did is the follwowing:

index="..."  | rex "^.{71}(?.*)"| spath input=data 

After this search I can access every attribute inside my data object and inside parameters and result, but I can't achieve writing both nested objects in my table.
However the objects parameter and result can greatly defer from one log line to another, also this is just a special log line, most of the log is containing only text.

Any idea how I can achieve this? Every help is highly appreciated.
Thanks,
Fabian

0 Karma
1 Solution

Digister
Explorer

Hi there,

after reading intensively the spath documentation I found a solution:

I used the following to get the complete parameters jobject:

index="c1006*" "- Summary" | rex "^.{71}(?.*)" | spath input=data| spath input=data path=parameters output=parameters | table _time, id, scenario, status, host, parameters

View solution in original post

0 Karma

Digister
Explorer

Hi there,

after reading intensively the spath documentation I found a solution:

I used the following to get the complete parameters jobject:

index="c1006*" "- Summary" | rex "^.{71}(?.*)" | spath input=data| spath input=data path=parameters output=parameters | table _time, id, scenario, status, host, parameters

0 Karma

coccyx
Path Finder

Why are you trying to count characters in your regex? This regex seems to do the trick: \s(?{.+})$.

In general, I'd try to get your developers to clean up their logging to output just JSON rather than JSON in a log. Another option to prettify the logs at ingestion time. It's a use case we're working on at Cribl (https://www.cribl.io/) if you're interested.

0 Karma

Digister
Explorer

I am not trying to count characters, I am removing 71 characters to only get the json data as input for spath, (rex "^.{71}(?.*)"). Therefore the regey you posted does not do the trick , but thanks for your comment! Unfortunately it is not going to happen, that they change the log. So my problem is still how can I access nested objects in general using spath?

0 Karma
Get Updates on the Splunk Community!

AppDynamics Summer Webinars

This summer, our mighty AppDynamics team is cooking up some delicious content on YouTube Live to satiate your ...

SOCin’ it to you at Splunk University

Splunk University is expanding its instructor-led learning portfolio with dedicated Security tracks at .conf25 ...

Credit Card Data Protection & PCI Compliance with Splunk Edge Processor

Organizations handling credit card transactions know that PCI DSS compliance is both critical and complex. The ...