Getting Data In

How to parse my sample JSON using spath?

jaireddy
Engager

I am trying to parse this json using spath, trying to create visualization from those kind of json logs. can some one help in parsing a query.

{
    "level": "debug",
    "message": "login_success_mla",
    "timestamp": "2017-07-05T20:50:27.358Z"
} {
    "transaction_type": "login",
    "username": "mla.dev1@somemail.com",
    "product": "mla",
    "x-pb-transactionid": "041c851f-69ef-4ea8-a5c8-2bdbf0813f32",
    "referer": "https://login-dev.saase2e.somecloud.com/mla?flow=v2&TargetResource=https%3A%2F%2Fsome.preview.com%2Foauth2%2Faus8zl82cdIGhGxGr0h7%2Fv1%2Fauthorize%3Fclient_id%3Dz1uxouBaavCLzSGOWfbD%26scope%3Dopenid%26response_type%3Did_token%20token%26redirect_uri%3Dhttp%3A%2F%2Flocalhost%3A8080%2F%26nonce%3D11111%26state%3D11111%26prompt%3Dnone%26response_mode%3Dfragment",
    "user-agent": "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:54.0) Gecko/20100101 Firefox/54.0",
    "transaction_status": "OK",
    "login_result": "success",
    "reason": "",
    "authn_status_code": "200",
    "message": "mla.dev1@somemail.com was able to authenticate successfully.",
    "version": "1.0.0.df85c67",
    "environment": "dev",
    "saase2eComponent": "LoginUI",
    "level": "info",
    "timestamp": "2017-07-05T20:50:27.358Z"
} {
    "level": "debug",
    "message": "https%3A%2F%2Fsome.apreview.com%2Foauth2%2Faus8zl82cdIGhGxGr0h7%2Fv1%2Fauthorize%3Fclient_id%3Dz1uxouBaavCLzSGOWfbD%26scope%3Dopenid%26response_type%3Did_token%20token%26redirect_uri%3Dhttp%3A%2F%2Flocalhost%3A8080%2F%26nonce%3D11111%26state%3D11111%26prompt%3Dnone%26response_mode%3Dfragment",
    "timestamp": "2017-07-05T20:50:27.358Z"
} {
    "username": "mla.dev1@somemail.com",
    "x-pb-transactionid": "041c851f-69ef-4ea8-a5c8-2bdbf0813f32",
    "transaction_type": "user_redirect",
    "redirectUrl": "https://some.apreview.com/oauth2/aus8zl82cdIGhGxGr0h7/v1/authorize?client_id=z1uxouBaavCLzSGOWfbD&scope=openid&response_type=id_token token&redirect_uri=http://localhost:8080/&nonce=11111&state=11111&prompt=none&response_mode=fragment&sessionToken=20111jMPScGMmAq6YaxA5Hc2p6wjU69qHmG3L01Z-ZrEpIi2fE636mH",
    "version": "1.0.0.df85c67",
    "environment": "dev",
    "saase2eComponent": "LoginUI",
    "level": "info",
    "message": "Redirecting user after successful login",
    "timestamp": "2017-07-05T20:50:27.358Z"
}
0 Karma

woodcock
Esteemed Legend

If you have split your events correctly so that the above string is 4 events, not just 1, then spath works just fine, as you can see with this:

| makeresults 
| eval raw=" {
     \"level\": \"debug\",
     \"message\": \"login_success_mla\",
     \"timestamp\": \"2017-07-05T20:50:27.358Z\"
 }::{
     \"transaction_type\": \"login\",
     \"username\": \"mla.dev1@somemail.com\",
     \"product\": \"mla\",
     \"x-pb-transactionid\": \"041c851f-69ef-4ea8-a5c8-2bdbf0813f32\",
     \"referer\": \"https://login-dev.saase2e.somecloud.com/mla?flow=v2&TargetResource=https%3A%2F%2Fsome.preview.com%2Foauth2%2Faus8zl82cdIGhGxGr0h7%2Fv1%2Fauthorize%3Fclient_id%3Dz1uxouBaavCLzSGOWfbD%26scope%3Dopenid%26response_type%3Did_token%20token%26redirect_uri%3Dhttp%3A%2F%2Flocalhost%3A8080%2F%26nonce%3D11111%26state%3D11111%26prompt%3Dnone%26response_mode%3Dfragment\",
     \"user-agent\": \"Mozilla/5.0 (Windows NT 10.0; WOW64; rv:54.0) Gecko/20100101 Firefox/54.0\",
     \"transaction_status\": \"OK\",
     \"login_result\": \"success\",
     \"reason\": \"\",
     \"authn_status_code\": \"200\",
     \"message\": \"mla.dev1@somemail.com was able to authenticate successfully.\",
     \"version\": \"1.0.0.df85c67\",
     \"environment\": \"dev\",
     \"saase2eComponent\": \"LoginUI\",
     \"level\": \"info\",
     \"timestamp\": \"2017-07-05T20:50:27.358Z\"
 }::{
     \"level\": \"debug\",
     \"message\":     \"https%3A%2F%2Fsome.apreview.com%2Foauth2%2Faus8zl82cdIGhGxGr0h7%2Fv1%2Fauthorize%3Fclient_id%3Dz1uxouBaavCLzSGOWfbD%26scope%3Dopenid%26response_type%3Did_token%20token%26redirect_uri%3Dhttp%3A%2F%2Flocalhost%3A8080%2F%26nonce%3D11111%26state%3D11111%26prompt%3Dnone%26response_mode%3Dfragment\",
     \"timestamp\": \"2017-07-05T20:50:27.358Z\"
 }::{
     \"username\": \"mla.dev1@somemail.com\",
     \"x-pb-transactionid\": \"041c851f-69ef-4ea8-a5c8-2bdbf0813f32\",
     \"transaction_type\": \"user_redirect\",
     \"redirectUrl\": \"https://some.apreview.com/oauth2/aus8zl82cdIGhGxGr0h7/v1/authorize?client_id=z1uxouBaavCLzSGOWfbD&scope=openid&response_type=id_token token&redirect_uri=http://localhost:8080/&nonce=11111&state=11111&prompt=none&response_mode=fragment&sessionToken=20111jMPScGMmAq6YaxA5Hc2p6wjU69qHmG3L01Z-ZrEpIi2fE636mH\",
     \"version\": \"1.0.0.df85c67\",
     \"environment\": \"dev\",
     \"saase2eComponent\": \"LoginUI\",
     \"level\": \"info\",
     \"message\": \"Redirecting user after successful login\",
     \"timestamp\": \"2017-07-05T20:50:27.358Z\"
 }"
| makemv delim="::" raw
| mvexpand raw
| rename raw AS _raw
| spath
0 Karma

woodcock
Esteemed Legend

Is this 4 events or just 1?

0 Karma

jaireddy
Engager

Thanks for the answer Woodcock. I have different kinds of Json log files, few logs have just one event, few have 2 and followed by 3 and max of 4 I guess, and when I validate these logs getting the validation errors and I have to make visualizations from this JSON log data with different structured format. Is that possible?

Thanks,
Jai

0 Karma

woodcock
Esteemed Legend

You really haven't answered my question. As my Answer here shows, IF the events are properly linebroken, then spath works just fine. So use LINE_BREAKER=[\r\n]+\}(\s+){[\r\n]+ and break them good.

0 Karma

jaireddy
Engager

This is just one log file, I think it is only 1 event?

0 Karma
Get Updates on the Splunk Community!

Why You Can't Miss .conf25: Unleashing the Power of Agentic AI with Splunk & Cisco

The Defining Technology Movement of Our Lifetime The advent of agentic AI is arguably the defining technology ...

Deep Dive into Federated Analytics: Unlocking the Full Power of Your Security Data

In today’s complex digital landscape, security teams face increasing pressure to protect sprawling data across ...

Your summer travels continue with new course releases

Summer in the Northern hemisphere is in full swing, and is often a time to travel and explore. If your summer ...