topic Re: find request but not in response in Splunk Search

find request but not in response

nmarun — Mon, 17 Jul 2023 13:28:48 GMT

Hi,

[12:30:13 INF 0ceafa153290582e1f1faec3f98d84ac] Gateway API|Request...
[12:30:15 INF 0ceafa153290582e1f1faec3f98d84ac] Gateway API|Response...

These are sample request, response structures that we log. There are scenarios where a request might not have a response. I'd like to write a query to find such correlation id (highlighted).

This is something that I tried, but it's fetching all the matches.

index=pcf sourcetype="gateway-api" "Gateway API|Request"
| rex field=msg "INF ?(?P<correlationId>[a-zA-Z0-9-_, ]*)]"
| rex field=msg "INF ?(?P<correlationIdReq>[a-zA-Z0-9-_, ]*)]"
| table correlationId, correlationIdReq
| join type="left" correlationId
[search index=pcf sourcetype="gateway-api" "Gateway API|Response"
| rex field=msg "INF ?(?P<correlationId>[a-zA-Z0-9-_, ]*)]"
| rex field=msg "INF ?(?P<correlationIdRes>[a-zA-Z0-9-_, ]*)]"
| table correlationId, correlationIdReq, correlationIdRes]

Thanks,

Arun

Re: find request but not in response

andrew_nelson — Mon, 17 Jul 2023 13:51:04 GMT

Try something like this:

Re: find request but not in response

nmarun — Mon, 17 Jul 2023 14:29:29 GMT

@andrew_nelson, thanks for the quick reply sir. I'd like to extract a few other fields from the request and response (different fields in both, as highlighted below).

I'm not sure how I can make your query do that work and that's why I created the structure that I have. Please let me know how I can get all these done.

Thanks,

Arun

Re: find request but not in response

andrew_nelson — Mon, 17 Jul 2023 14:53:33 GMT

You can write two custom extractions, one for each log type. Some rough extractions below.
The request extraction:

| rex field=msg "Gateway API\|Request\|(?<method>[^\|]+)\|(?<field2>[^\|]+)\|(?<field3>[^\|]+)\|(?<field4>[^\|]+)\|(?<field5>[^\|]+)\|..."

The response extraction:

| rex field=msg "Gateway API\|Response\|(?<response_code>\d+)\|(?<field2>[^\|]+)\|(?<field3>[^\|]+)\|(?<field4>[^\|]+)\|(?<field5>[^\|]+)\|..."

So assuming all of your fields are pipe separated ( | ) these extractions, will pull fields out for everything between the pipe symbol. You can rename the field2, field3 etc to what they actually are.

If your dataset is actual event data, then its best to create those into proper field extractions that don't need to be created for every individual search.

Re: find request but not in response

PickleRick — Mon, 17 Jul 2023 16:01:23 GMT

First and foremost - don't use the "join" command unless you absolutely cannot avoid it.

In this case there's another command which you could use but it's also a "bad one" - it's "transaction".

But I supposed the "lightest" solution here would be to extract the correlationId as you did, extract the type of the call (Request/Response) and then do

| stats values(call) by correlationId

Then you can find only those which do not have responses

| where NOT call="Response"

(mind you that it's not the same as call!="Response")

Of course if you want to get some details of the requests, there will be more magic needed.

Re: find request but not in response

nmarun — Tue, 18 Jul 2023 05:27:17 GMT

Shoot.. I should've thought of that.

Thanks @andrew_nelson.