Splunk Search

How to extract value from log events with type as json object or json array??

bhavika100
Explorer

Our event log has request and response. Request and response body can either be a json object or json array. I need to extract resquest.body and response.body to construct a field "httpdetails" which is a string . How can i achieve this using single spath function.

example of log events :

 

 

{
  "message": {
    "request": {
      "body": {}
    },
    "response": {
      "body": [
        {
          "id": "85118db6-2d5c-6bb0-ff67-5bc9ef5d4a1f",
          "createdon": "2021-07-08T00:37:02.512Z"
        }
      ]
    }
  }
}

 

 

 

 

{
  "message": {
    "request": {
      "body": {
        "$limitafter": "2021-07-08T20:08:29.983Z"
      }
    },
    "response": {
      "statuscode": 200,
      "body": {
        "count": "22"
      }
    }
  }
}

 

 

Splunk query :

| spath output=response_data message.response.body | spath output=request_data message.request.body | eval request_data=if(isnull(request_data) , NULL , request_data) | eval response_data=if(isnull(response_data),  NULL, response_data) | eval httpdetails="\n"+request_data+"\n-----------------Response---------------\n"+response_data, httpdetails = split(httpdetails,"\n") | eval details=if(isnotnull(httpdetails), httpdetails, details) 

After running this query "httpdetails" is shown below. Here response_data for first log event is coming as NULL instead of object array. How can I fix this??

Screen Shot 2021-07-12 at 3.00.50 PM.png

 

Labels (1)
0 Karma
1 Solution

venkatasri
SplunkTrust
SplunkTrust

Hi @bhavika100 

Can you try this, I would have done this much cleaner however your json payload is so dynamic with arrays and field names change.

| eval req=replace(json_extract(_raw, "message.request"),"body","") 
| eval res=replace(json_extract(_raw, "message.response"),"body","") 
| eval httpdetails="\n"+req+"\n-----------------Response---------------\n"+res, httpdetails = split(httpdetails,"\n") 
| eval details=if(isnotnull(httpdetails), httpdetails, details) 
| table details

 You can further truncate {" if you want much cleaner, response does come-up with this SPL.

venkatasri_0-1626152672194.png

---

An upvote would be appreciated and Accept Solution if this reply helps!

View solution in original post

0 Karma

venkatasri
SplunkTrust
SplunkTrust

Hi @bhavika100 

Can you try this, I would have done this much cleaner however your json payload is so dynamic with arrays and field names change.

| eval req=replace(json_extract(_raw, "message.request"),"body","") 
| eval res=replace(json_extract(_raw, "message.response"),"body","") 
| eval httpdetails="\n"+req+"\n-----------------Response---------------\n"+res, httpdetails = split(httpdetails,"\n") 
| eval details=if(isnotnull(httpdetails), httpdetails, details) 
| table details

 You can further truncate {" if you want much cleaner, response does come-up with this SPL.

venkatasri_0-1626152672194.png

---

An upvote would be appreciated and Accept Solution if this reply helps!

0 Karma

bhavika100
Explorer

Hi @venkatasri Thanks for the quick resposne. This solution works fine for the above logs. I tried to implement the same in my query but fails when either request or response body is null. Log event is as below.

log event:

{
  "message": {
    "request": {
      "body": null
    },
    "response": {
      "statuscode": 200,
      "body": {
        "id": "e4214ec1-3d16-6083-ec11-beb01188ddaf"
      }
    }
  }
}

details is coming as empty for this log event.

0 Karma

venkatasri
SplunkTrust
SplunkTrust

@bhavika100 Sure thing let me find out.

0 Karma

venkatasri
SplunkTrust
SplunkTrust

@bhavika100 I have tried your new payload there message.request.body = null hence same has been showing in output as null below and response is fine. 

venkatasri_0-1626397674062.png

---

An upvote would be appreciated if this reply helps!

 

bhavika100
Explorer

This works!! 

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...