My current splunk search stops after 5 errors of "Streamed search execute failed because: Error in 'rex' command: ". It does say "exceeded configured match_limit" so i think on some log lines this execution take too much steps. Now up till here it's pretty clear to me, but it does succeed for thousands to millions of lines before stopping completely. i would like to know which line it fails on so i can investigate the problem and fix it. Does anyone know how to investigate?
the query:
the error is reported on the regex line with "ExecutionContext", line 3
host="abc*-solutionname-*" "Elapsed:" "Finished" "ExecutionContext:"
| rex "Elapsed: (?<elapsed>[^\s]+)"
| rex "ExecutionContext:(?<method>.*)Elapsed:"
| eval stamp=strptime(elapsed, "%H:%M:%S.%Q.")-strptime(strftime(now(),"%m/%d/%y"),"%m/%d/%y")
A few example lines being processed:
2020-10-30 12:15:37.0373Z|TRACE|ServiceLog|UID:[]. CID:[fb749af30b9e2b]. PID:[15]. SID:[4cfda0137]. ExecutionContext:Help.Index. Elapsed: 00:00:00.0006980. Finished. Message: blabla.|
2020-10-30 12:15:37.0373Z|TRACE|ServiceLog|UID:[]. CID:[fb749af30b9e2b]. PID:[15]. SID:[4cfda0137]. ExecutionContext:HelpController.Index. Elapsed: 00:00:00.0000061. Finished. |
2020-10-30 12:15:37.0287Z|TRACE|ServiceLog|UID:[]. CID:[5b0f8455873d1a]. PID:[10]. SID:[d41870240]. ExecutionContext:Help.Index. Elapsed: 00:00:00.0007222. Finished. Message: blabla.|
2020-10-30 12:15:37.0287Z|TRACE|ServiceLog|UID:[]. CID:[5b0f8455873d1a]. PID:[10]. SID:[d41870240]. ExecutionContext:HelpController.Index. Elapsed: 00:00:00.0000061. Finished. |
i do know THESE lines work fine, but of course there are quite some different possibilities, so i would like to know when it goes wrong EXACTLY instead of those "yeah something went wrong and i'm not telling you where" errors.
