How to parse my sample JSON using spath?

[jaireddy]

I am trying to parse this json using spath, trying to create visualization from those kind of json logs. can some one help in parsing a query.

{
    "level": "debug",
    "message": "login_success_mla",
    "timestamp": "2017-07-05T20:50:27.358Z"
} {
    "transaction_type": "login",
    "username": "[email protected]",
    "product": "mla",
    "x-pb-transactionid": "041c851f-69ef-4ea8-a5c8-2bdbf0813f32",
    "referer": "https://login-dev.saase2e.somecloud.com/mla?flow=v2&TargetResource=https%3A%2F%2Fsome.preview.com%2Foauth2%2Faus8zl82cdIGhGxGr0h7%2Fv1%2Fauthorize%3Fclient_id%3Dz1uxouBaavCLzSGOWfbD%26scope%3Dopenid%26response_type%3Did_token%20token%26redirect_uri%3Dhttp%3A%2F%2Flocalhost%3A8080%2F%26nonce%3D11111%26state%3D11111%26prompt%3Dnone%26response_mode%3Dfragment",
    "user-agent": "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:54.0) Gecko/20100101 Firefox/54.0",
    "transaction_status": "OK",
    "login_result": "success",
    "reason": "",
    "authn_status_code": "200",
    "message": "[email protected] was able to authenticate successfully.",
    "version": "1.0.0.df85c67",
    "environment": "dev",
    "saase2eComponent": "LoginUI",
    "level": "info",
    "timestamp": "2017-07-05T20:50:27.358Z"
} {
    "level": "debug",
    "message": "https%3A%2F%2Fsome.apreview.com%2Foauth2%2Faus8zl82cdIGhGxGr0h7%2Fv1%2Fauthorize%3Fclient_id%3Dz1uxouBaavCLzSGOWfbD%26scope%3Dopenid%26response_type%3Did_token%20token%26redirect_uri%3Dhttp%3A%2F%2Flocalhost%3A8080%2F%26nonce%3D11111%26state%3D11111%26prompt%3Dnone%26response_mode%3Dfragment",
    "timestamp": "2017-07-05T20:50:27.358Z"
} {
    "username": "[email protected]",
    "x-pb-transactionid": "041c851f-69ef-4ea8-a5c8-2bdbf0813f32",
    "transaction_type": "user_redirect",
    "redirectUrl": "https://some.apreview.com/oauth2/aus8zl82cdIGhGxGr0h7/v1/authorize?client_id=z1uxouBaavCLzSGOWfbD&scope=openid&response_type=id_token token&redirect_uri=http://localhost:8080/&nonce=11111&state=11111&prompt=none&response_mode=fragment&sessionToken=20111jMPScGMmAq6YaxA5Hc2p6wjU69qHmG3L01Z-ZrEpIi2fE636mH",
    "version": "1.0.0.df85c67",
    "environment": "dev",
    "saase2eComponent": "LoginUI",
    "level": "info",
    "message": "Redirecting user after successful login",
    "timestamp": "2017-07-05T20:50:27.358Z"
}

[woodcock]

If you have split your events correctly so that the above string is 4 events, not just 1, then spath works just fine, as you can see with this:

| makeresults 
| eval raw=" {
     \"level\": \"debug\",
     \"message\": \"login_success_mla\",
     \"timestamp\": \"2017-07-05T20:50:27.358Z\"
 }::{
     \"transaction_type\": \"login\",
     \"username\": \"[email protected]\",
     \"product\": \"mla\",
     \"x-pb-transactionid\": \"041c851f-69ef-4ea8-a5c8-2bdbf0813f32\",
     \"referer\": \"https://login-dev.saase2e.somecloud.com/mla?flow=v2&TargetResource=https%3A%2F%2Fsome.preview.com%2Foauth2%2Faus8zl82cdIGhGxGr0h7%2Fv1%2Fauthorize%3Fclient_id%3Dz1uxouBaavCLzSGOWfbD%26scope%3Dopenid%26response_type%3Did_token%20token%26redirect_uri%3Dhttp%3A%2F%2Flocalhost%3A8080%2F%26nonce%3D11111%26state%3D11111%26prompt%3Dnone%26response_mode%3Dfragment\",
     \"user-agent\": \"Mozilla/5.0 (Windows NT 10.0; WOW64; rv:54.0) Gecko/20100101 Firefox/54.0\",
     \"transaction_status\": \"OK\",
     \"login_result\": \"success\",
     \"reason\": \"\",
     \"authn_status_code\": \"200\",
     \"message\": \"[email protected] was able to authenticate successfully.\",
     \"version\": \"1.0.0.df85c67\",
     \"environment\": \"dev\",
     \"saase2eComponent\": \"LoginUI\",
     \"level\": \"info\",
     \"timestamp\": \"2017-07-05T20:50:27.358Z\"
 }::{
     \"level\": \"debug\",
     \"message\":     \"https%3A%2F%2Fsome.apreview.com%2Foauth2%2Faus8zl82cdIGhGxGr0h7%2Fv1%2Fauthorize%3Fclient_id%3Dz1uxouBaavCLzSGOWfbD%26scope%3Dopenid%26response_type%3Did_token%20token%26redirect_uri%3Dhttp%3A%2F%2Flocalhost%3A8080%2F%26nonce%3D11111%26state%3D11111%26prompt%3Dnone%26response_mode%3Dfragment\",
     \"timestamp\": \"2017-07-05T20:50:27.358Z\"
 }::{
     \"username\": \"[email protected]\",
     \"x-pb-transactionid\": \"041c851f-69ef-4ea8-a5c8-2bdbf0813f32\",
     \"transaction_type\": \"user_redirect\",
     \"redirectUrl\": \"https://some.apreview.com/oauth2/aus8zl82cdIGhGxGr0h7/v1/authorize?client_id=z1uxouBaavCLzSGOWfbD&scope=openid&response_type=id_token token&redirect_uri=http://localhost:8080/&nonce=11111&state=11111&prompt=none&response_mode=fragment&sessionToken=20111jMPScGMmAq6YaxA5Hc2p6wjU69qHmG3L01Z-ZrEpIi2fE636mH\",
     \"version\": \"1.0.0.df85c67\",
     \"environment\": \"dev\",
     \"saase2eComponent\": \"LoginUI\",
     \"level\": \"info\",
     \"message\": \"Redirecting user after successful login\",
     \"timestamp\": \"2017-07-05T20:50:27.358Z\"
 }"
| makemv delim="::" raw
| mvexpand raw
| rename raw AS _raw
| spath

[woodcock]

Is this 4 events or just 1?

[jaireddy]

Thanks for the answer Woodcock. I have different kinds of Json log files, few logs have just one event, few have 2 and followed by 3 and max of 4 I guess, and when I validate these logs getting the validation errors and I have to make visualizations from this JSON log data with different structured format. Is that possible?

Thanks,
Jai

[woodcock]

You really haven't answered my question. As my Answer here shows, IF the events are properly linebroken, then spath works just fine. So use LINE_BREAKER=[\r\n]+\}(\s+){[\r\n]+ and break them good.

[jaireddy]

This is just one log file, I think it is only 1 event?