topic Re: How to extract 2 fields in same "rex"? in Splunk Search

How to extract 2 fields in same "rex"?

dickersons — Thu, 16 Feb 2023 21:02:02 GMT

Hi,

I have a search where I am attempting to extracting 2 different fields from one string response using "rex":

1st Field: rex \"traceId\"\s:\s\"?(?<traceId>.*?)\" 2nd Field: rex "\"statusCode\"\s:\s\"?(?<tstatusCode>2\d{2}|4\d{2}|5\d{2})\"?"

I am attempting to "dedup" the 1st field (traceId) before I pipe those results into the 2nd field (statusCode). I have attempted multiple variation based on Splunk threads and other internet resources. Below is the query I am making:

index=myCoolIndex cluster_name="myCoolCluster" sourcetype=myCoolSourceType label_app=myCoolApp ("\"statusCode\"") | rex \"traceId\"\s:\s\"?(?<traceId>.*?)\" | dedup traceId | rex "\"statusCode\"\s:\s\"?(?<tstatusCode>2\d{2}|4\d{2}|5\d{2})\"?" //I have tried a lot of other permutations this is just one

Below is the response from the log (looks like JSON but it is string type):

\\Sample Log (Looks like JSON object, but its a string): "{ "correlationId" : "", "message" : "", "tracePoint" : "", "priority" : "", "category" : "", "elapsed" : 0, "locationInfo" : { "lineInFile" : "", "component" : "", "fileName" : "", "rootContainer" : "" }, "timestamp" : "", "content" : { "message" : "", "originalError" : { "statusCode" : "200", "errorPayload" : { "error" : "" } }, "standardizedError" : { "statusCode" : "500", "errorPayload" : { "errors" : [ { "error" : { "traceId" : "9539510-d8771da0-a7ce-11ed-921c-d6a73926c0ac", "errorCode" : "", "errorDescription" : "" "errorDetails" : "" } } ] } } }, }"

The intent of the query is to:

Extract field "traceId", then "dedup" "traceId" (to remove duplicates), then extract field "statusCode" and sort "statusCode" values.

When running these regEx's independently of eachother they work as expected, but I need to combine them into one query as I will be creating charts on my next step..... All help is appreciated.

Re: How to extract 2 fields in same "rex"?

ITWhisperer — Thu, 16 Feb 2023 21:41:34 GMT

| rex "\"statusCode\"\s:\s\"?(?<statusCode>2\d{2}|4\d{2}|5\d{2})\".*?\"traceId\"\s:\s\"?(?<traceId>.*?)\"" | dedup traceId

Re: How to extract 2 fields in same "rex"?

dickersons — Thu, 16 Feb 2023 21:49:41 GMT

This does not produce a match in Splunk or regex101.......

Re: How to extract 2 fields in same "rex"?

bowesmana — Fri, 17 Feb 2023 03:34:26 GMT

This will get the _first_ statuscode (200)

| rex "(?s)\"statusCode\"\s:\s\"?(?<statusCode>2\d{2}|4\d{2}|5\d{2})\"?.*\"traceId\"\s:\s\"?(?<traceId>.*?)\".*"

The (?s) allows .* to span multiple lines (. = LF matching).

This will get the _last_ status code

| rex max_match=0 "(?s).*\"statusCode\"\s:\s\"?(?<statusCode>[245]\d{2})\"?.*\"traceId\"\s:\s\"?(?<traceId>.*?)\""

i.e. by starting the regex with a greedy .* before the statusCode it will consume everything up to the last occurrence of statusCode, that exists BEFORE the traceId

Re: How to extract 2 fields in same "rex"?

ITWhisperer — Fri, 17 Feb 2023 06:26:15 GMT

| rex "(?ms)\"statusCode\"\s:\s\"?(?<statusCode>2\d{2}|4\d{2}|5\d{2})\".*?\"traceId\"\s:\s\"?(?<traceId>.*?)\"" | dedup traceId

Re: How to extract 2 fields in same "rex"?

dickersons — Fri, 17 Feb 2023 20:52:40 GMT

Works as stated above. Thanks much for the assistance!