Splunk Search

How to extract value from log events with type as json object or json array??

bhavika100
Explorer

Our event log has request and response. Request and response body can either be a json object or json array. I need to extract resquest.body and response.body to construct a field "httpdetails" which is a string . How can i achieve this using single spath function.

example of log events :

 

 

{
  "message": {
    "request": {
      "body": {}
    },
    "response": {
      "body": [
        {
          "id": "85118db6-2d5c-6bb0-ff67-5bc9ef5d4a1f",
          "createdon": "2021-07-08T00:37:02.512Z"
        }
      ]
    }
  }
}

 

 

 

 

{
  "message": {
    "request": {
      "body": {
        "$limitafter": "2021-07-08T20:08:29.983Z"
      }
    },
    "response": {
      "statuscode": 200,
      "body": {
        "count": "22"
      }
    }
  }
}

 

 

Splunk query :

| spath output=response_data message.response.body | spath output=request_data message.request.body | eval request_data=if(isnull(request_data) , NULL , request_data) | eval response_data=if(isnull(response_data),  NULL, response_data) | eval httpdetails="\n"+request_data+"\n-----------------Response---------------\n"+response_data, httpdetails = split(httpdetails,"\n") | eval details=if(isnotnull(httpdetails), httpdetails, details) 

After running this query "httpdetails" is shown below. Here response_data for first log event is coming as NULL instead of object array. How can I fix this??

Screen Shot 2021-07-12 at 3.00.50 PM.png

 

Labels (1)
0 Karma
1 Solution

venkatasri
SplunkTrust
SplunkTrust

Hi @bhavika100 

Can you try this, I would have done this much cleaner however your json payload is so dynamic with arrays and field names change.

| eval req=replace(json_extract(_raw, "message.request"),"body","") 
| eval res=replace(json_extract(_raw, "message.response"),"body","") 
| eval httpdetails="\n"+req+"\n-----------------Response---------------\n"+res, httpdetails = split(httpdetails,"\n") 
| eval details=if(isnotnull(httpdetails), httpdetails, details) 
| table details

 You can further truncate {" if you want much cleaner, response does come-up with this SPL.

venkatasri_0-1626152672194.png

---

An upvote would be appreciated and Accept Solution if this reply helps!

View solution in original post

0 Karma

venkatasri
SplunkTrust
SplunkTrust

Hi @bhavika100 

Can you try this, I would have done this much cleaner however your json payload is so dynamic with arrays and field names change.

| eval req=replace(json_extract(_raw, "message.request"),"body","") 
| eval res=replace(json_extract(_raw, "message.response"),"body","") 
| eval httpdetails="\n"+req+"\n-----------------Response---------------\n"+res, httpdetails = split(httpdetails,"\n") 
| eval details=if(isnotnull(httpdetails), httpdetails, details) 
| table details

 You can further truncate {" if you want much cleaner, response does come-up with this SPL.

venkatasri_0-1626152672194.png

---

An upvote would be appreciated and Accept Solution if this reply helps!

0 Karma

bhavika100
Explorer

Hi @venkatasri Thanks for the quick resposne. This solution works fine for the above logs. I tried to implement the same in my query but fails when either request or response body is null. Log event is as below.

log event:

{
  "message": {
    "request": {
      "body": null
    },
    "response": {
      "statuscode": 200,
      "body": {
        "id": "e4214ec1-3d16-6083-ec11-beb01188ddaf"
      }
    }
  }
}

details is coming as empty for this log event.

0 Karma

venkatasri
SplunkTrust
SplunkTrust

@bhavika100 Sure thing let me find out.

0 Karma

venkatasri
SplunkTrust
SplunkTrust

@bhavika100 I have tried your new payload there message.request.body = null hence same has been showing in output as null below and response is fine. 

venkatasri_0-1626397674062.png

---

An upvote would be appreciated if this reply helps!

 

bhavika100
Explorer

This works!! 

0 Karma
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...