So I have a single log event that captures the request and the response JSONs. As a user I'd like to be able to write a query that will capture the fields from the JSONs, but the field names are the same in the request and the response, so when I search:
index="myIndex" sourcetype="mySourceType" "Keywords to search for only request and response events" |
rex field=_raw "This event received the following request (?<requestJson>.*) and sent the following response (?<responseJson>.*)" |
spath input=requestJson |
spath input=responseJson
When I get the results of this search, I get one field with two values (request and response values):
"clientId":[123, 123] <-----searched by
"name":[null, "Joe Schmoe"]
"ssn":[null, "123-45-6789"]
.....etc.
What I'd really like to be able to do is get a response more like:
"request.clientId":123
"request.name":null
"request.ssn":null
"response.clientId":123
"response.name":"Joe Schmoe"
"response.ssn":"123-45-6789"
I tried renaming the fields in "requestJson" after using spath:
spath input=requestJson | rename * as request.*
but that doesn't seem to work unless I use at least one letter before the wildcard (*), such as:
spath input=requestJson | rename a* as request.*
How can I rename these fields generated dynamically by spath-ing my JSONs? Or, alternative I may be missing: how can I differentiate between the request and response values even though they have the same field name?
@seomaniv add the following eval before spath
commands.
| eval requestJson="{\"request\":".requestJson."\}", responseJson="{\"response\":".responseJson."\}"
Following is a run anywhere search example based on the sample data provided:
| makeresults
| eval _raw="This event received the following request {\"clientId\":123,\"name\":null,\"ssn\":null} and sent the following response {\"clientId\":123,\"name\":\"John\",\"ssn\":\"Doe\"}"
| rex "This event received the following request (?<requestJson>.*) and sent the following response (?<responseJson>.*)"
| eval requestJson="{\"request\":".requestJson."\}", responseJson="{\"response\":".responseJson."\}"
| spath input=requestJson
| spath input=responseJson
| fields - _raw requestJson responseJson
| fields request* response*
Please try out and confirm!
@seomaniv add the following eval before spath
commands.
| eval requestJson="{\"request\":".requestJson."\}", responseJson="{\"response\":".responseJson."\}"
Following is a run anywhere search example based on the sample data provided:
| makeresults
| eval _raw="This event received the following request {\"clientId\":123,\"name\":null,\"ssn\":null} and sent the following response {\"clientId\":123,\"name\":\"John\",\"ssn\":\"Doe\"}"
| rex "This event received the following request (?<requestJson>.*) and sent the following response (?<responseJson>.*)"
| eval requestJson="{\"request\":".requestJson."\}", responseJson="{\"response\":".responseJson."\}"
| spath input=requestJson
| spath input=responseJson
| fields - _raw requestJson responseJson
| fields request* response*
Please try out and confirm!
Actually I ended up figuring it out, too. What I did was concatenate both fields into a single field, then ran spath on that field and it did the work itself.
eval toSpath="{\"request\":".requestJson.",\"response\":".responseJson |
spath input=toSpath
Same thing you did, basically. Thanks niketnilay!
@seomaniv ,Anytime! Glad you figured it out 🙂