Getting Data In

How to parse my sample JSON using spath?

jaireddy
Engager

I am trying to parse this json using spath, trying to create visualization from those kind of json logs. can some one help in parsing a query.

{
    "level": "debug",
    "message": "login_success_mla",
    "timestamp": "2017-07-05T20:50:27.358Z"
} {
    "transaction_type": "login",
    "username": "mla.dev1@somemail.com",
    "product": "mla",
    "x-pb-transactionid": "041c851f-69ef-4ea8-a5c8-2bdbf0813f32",
    "referer": "https://login-dev.saase2e.somecloud.com/mla?flow=v2&TargetResource=https%3A%2F%2Fsome.preview.com%2Foauth2%2Faus8zl82cdIGhGxGr0h7%2Fv1%2Fauthorize%3Fclient_id%3Dz1uxouBaavCLzSGOWfbD%26scope%3Dopenid%26response_type%3Did_token%20token%26redirect_uri%3Dhttp%3A%2F%2Flocalhost%3A8080%2F%26nonce%3D11111%26state%3D11111%26prompt%3Dnone%26response_mode%3Dfragment",
    "user-agent": "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:54.0) Gecko/20100101 Firefox/54.0",
    "transaction_status": "OK",
    "login_result": "success",
    "reason": "",
    "authn_status_code": "200",
    "message": "mla.dev1@somemail.com was able to authenticate successfully.",
    "version": "1.0.0.df85c67",
    "environment": "dev",
    "saase2eComponent": "LoginUI",
    "level": "info",
    "timestamp": "2017-07-05T20:50:27.358Z"
} {
    "level": "debug",
    "message": "https%3A%2F%2Fsome.apreview.com%2Foauth2%2Faus8zl82cdIGhGxGr0h7%2Fv1%2Fauthorize%3Fclient_id%3Dz1uxouBaavCLzSGOWfbD%26scope%3Dopenid%26response_type%3Did_token%20token%26redirect_uri%3Dhttp%3A%2F%2Flocalhost%3A8080%2F%26nonce%3D11111%26state%3D11111%26prompt%3Dnone%26response_mode%3Dfragment",
    "timestamp": "2017-07-05T20:50:27.358Z"
} {
    "username": "mla.dev1@somemail.com",
    "x-pb-transactionid": "041c851f-69ef-4ea8-a5c8-2bdbf0813f32",
    "transaction_type": "user_redirect",
    "redirectUrl": "https://some.apreview.com/oauth2/aus8zl82cdIGhGxGr0h7/v1/authorize?client_id=z1uxouBaavCLzSGOWfbD&scope=openid&response_type=id_token token&redirect_uri=http://localhost:8080/&nonce=11111&state=11111&prompt=none&response_mode=fragment&sessionToken=20111jMPScGMmAq6YaxA5Hc2p6wjU69qHmG3L01Z-ZrEpIi2fE636mH",
    "version": "1.0.0.df85c67",
    "environment": "dev",
    "saase2eComponent": "LoginUI",
    "level": "info",
    "message": "Redirecting user after successful login",
    "timestamp": "2017-07-05T20:50:27.358Z"
}
0 Karma

woodcock
Esteemed Legend

If you have split your events correctly so that the above string is 4 events, not just 1, then spath works just fine, as you can see with this:

| makeresults 
| eval raw=" {
     \"level\": \"debug\",
     \"message\": \"login_success_mla\",
     \"timestamp\": \"2017-07-05T20:50:27.358Z\"
 }::{
     \"transaction_type\": \"login\",
     \"username\": \"mla.dev1@somemail.com\",
     \"product\": \"mla\",
     \"x-pb-transactionid\": \"041c851f-69ef-4ea8-a5c8-2bdbf0813f32\",
     \"referer\": \"https://login-dev.saase2e.somecloud.com/mla?flow=v2&TargetResource=https%3A%2F%2Fsome.preview.com%2Foauth2%2Faus8zl82cdIGhGxGr0h7%2Fv1%2Fauthorize%3Fclient_id%3Dz1uxouBaavCLzSGOWfbD%26scope%3Dopenid%26response_type%3Did_token%20token%26redirect_uri%3Dhttp%3A%2F%2Flocalhost%3A8080%2F%26nonce%3D11111%26state%3D11111%26prompt%3Dnone%26response_mode%3Dfragment\",
     \"user-agent\": \"Mozilla/5.0 (Windows NT 10.0; WOW64; rv:54.0) Gecko/20100101 Firefox/54.0\",
     \"transaction_status\": \"OK\",
     \"login_result\": \"success\",
     \"reason\": \"\",
     \"authn_status_code\": \"200\",
     \"message\": \"mla.dev1@somemail.com was able to authenticate successfully.\",
     \"version\": \"1.0.0.df85c67\",
     \"environment\": \"dev\",
     \"saase2eComponent\": \"LoginUI\",
     \"level\": \"info\",
     \"timestamp\": \"2017-07-05T20:50:27.358Z\"
 }::{
     \"level\": \"debug\",
     \"message\":     \"https%3A%2F%2Fsome.apreview.com%2Foauth2%2Faus8zl82cdIGhGxGr0h7%2Fv1%2Fauthorize%3Fclient_id%3Dz1uxouBaavCLzSGOWfbD%26scope%3Dopenid%26response_type%3Did_token%20token%26redirect_uri%3Dhttp%3A%2F%2Flocalhost%3A8080%2F%26nonce%3D11111%26state%3D11111%26prompt%3Dnone%26response_mode%3Dfragment\",
     \"timestamp\": \"2017-07-05T20:50:27.358Z\"
 }::{
     \"username\": \"mla.dev1@somemail.com\",
     \"x-pb-transactionid\": \"041c851f-69ef-4ea8-a5c8-2bdbf0813f32\",
     \"transaction_type\": \"user_redirect\",
     \"redirectUrl\": \"https://some.apreview.com/oauth2/aus8zl82cdIGhGxGr0h7/v1/authorize?client_id=z1uxouBaavCLzSGOWfbD&scope=openid&response_type=id_token token&redirect_uri=http://localhost:8080/&nonce=11111&state=11111&prompt=none&response_mode=fragment&sessionToken=20111jMPScGMmAq6YaxA5Hc2p6wjU69qHmG3L01Z-ZrEpIi2fE636mH\",
     \"version\": \"1.0.0.df85c67\",
     \"environment\": \"dev\",
     \"saase2eComponent\": \"LoginUI\",
     \"level\": \"info\",
     \"message\": \"Redirecting user after successful login\",
     \"timestamp\": \"2017-07-05T20:50:27.358Z\"
 }"
| makemv delim="::" raw
| mvexpand raw
| rename raw AS _raw
| spath
0 Karma

woodcock
Esteemed Legend

Is this 4 events or just 1?

0 Karma

jaireddy
Engager

Thanks for the answer Woodcock. I have different kinds of Json log files, few logs have just one event, few have 2 and followed by 3 and max of 4 I guess, and when I validate these logs getting the validation errors and I have to make visualizations from this JSON log data with different structured format. Is that possible?

Thanks,
Jai

0 Karma

woodcock
Esteemed Legend

You really haven't answered my question. As my Answer here shows, IF the events are properly linebroken, then spath works just fine. So use LINE_BREAKER=[\r\n]+\}(\s+){[\r\n]+ and break them good.

0 Karma

jaireddy
Engager

This is just one log file, I think it is only 1 event?

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Get More Out of Your Security Practice With a SIEM

Get More Out of Your Security Practice With a SIEMWednesday, July 31, 2024  |  11AM PT / 2PM ETREGISTER ...