Rex has exceeded configured match_limit, consider raising the value in limits.conf.

[ssh]

In our log, I'd like to extract statusText and categorize it in table to see how many error response statusCode and statusText.
EX:

eventSource	statusCode	statusText
bulkDelete	1020	3031: No Card found with the identifier for the request

But my query is getting "has exceeded configured match_limit, consider raising the value in limits.conf." after using fields extraction.

index = xxx sourcetype=xxx "Publish message on SQS" | search bulkDelete | rex field=_raw "(?ms)^(?:[^:\\n]*:){7}\"(?P<error_bulkDelete>[^\"]+)(?:[^:\\n]*:){2}\"(?P<error_errorCode>[^\"]+)[^:\\n]*:\"(?P<error_desc>[^\"]+)(?:[^:\\n]*:){6}\\\\\"(?P<error_statusText>[^\\\\]+)" offset_field=_extracted_fields_bounds

Target log:

Publish message on SQS, queueName=xxx, retryCount=0, message={"traceId":"xxx1112233","clientContext":"xxxxxclientContext","cardTokenReferenceId":"xxxcardTokenReferenceId","eventSource":"bulkDelete","errors":[{"errorCode":"52099","errorDescription":"Feign Client Exception.","retryCategory":"RETRYABLE","errorDetails":"{\"clientContext\":\"xxxxxclientContext\",\"ewSID\":\"xxxxSID\",\"statusCode\":\"1020\",\"statusText\":\"3031: No Card found with the identifier for the request\",\"timestampISO8601\":\"2024-04-05T00:00:26Z\"}"}]}

I checked similar posts, they suggested to use non-greedy?

So I tried:

index = "xxx" sourcetype=xxx "Publish message on SQS*" bulkDelete | rex field=_raw "\"statusText\":\s*\"(?P<statusText>[^\"]+)\"" | where NOT LIKE( statusText, "%Success%")

If I add "| table", I will get blank content on statusText

[marnall · Accepted solution]

Yes you could exclude successful responses by adding a filter. Assuming that all errors have an errorCode and all non-errors do not, then you could do it like this:

index = xxx sourcetype=xxx "Publish message on SQS" bulkDelete
| rex field=_raw "message=(?<message>{.*}$)"
| spath input=message
| search "errors{}.errorCode" = *
| spath input=errors{}.errorDetails
| table eventSource statusCode statusText

[marnall]

You could try extracting the json object after message=, then spathing it until you get the fields you would like. E.g.

index = xxx sourcetype=xxx "Publish message on SQS" bulkDelete
| rex field=_raw "message=(?<message>{.*}$)"
| spath input=message
| spath input=errors{}.errorDetails
| table eventSource statusCode statusText

[ssh]

Thanks, it looks like contain successful response, can we exclude it?

Publish message on SQS, queueName=xxx, retryCount=0, message={"traceId":"xxxtraceId","clientContext":"xxxclientContext","cardTokenReferenceId":"xxxCardTokenReferenceId","eventSource":"bulkDelete","walletWebResponse":{"clientContext":"xxxclientContext","ewSID":"xxxSID,"timestampISO8601":"2024-04-05T00:00:14Z","statusCode":"0","statusText":"Success"}}