topic Re: How to optimize rex to avoid the error message: Error in 'rex' command: regex= has exceeded configured match_limit, consider raising the value in limits.conf in Splunk Search

How to optimize rex to avoid the error message: Error in 'rex' command: regex= has exceeded configured match_limit, consider raising the value in limits.conf

spisiakmi — Fri, 06 Sep 2019 12:14:20 GMT

Hi. Can you help me, please, to optimize the regular expression. The problem is, when I search in longer time, I receive the error message: Error in 'rex' command: regex= has exceeded configured match_limit, consider raising the value in limits.conf
I do not want to adjust the limits.conf, I want to write proper regex.
The search code has been uploaded as image search.jpg

The example of the xml log file has been uploaded as an image regex_prob.jpg.

I want to read the whole section which belongs to the "test".

Re: How to optimize rex to avoid the error message: Error in 'rex' command: regex= has exceeded configured match_limit, consider raising the value in limits.conf

harsmarvania57 — Fri, 06 Sep 2019 14:28:45 GMT

Is it possible you to provide sample data in text instead of image (Please mask any sensitive data) ?

Re: How to optimize rex to avoid the error message: Error in 'rex' command: regex= has exceeded configured match_limit, consider raising the value in limits.conf

spisiakmi — Mon, 09 Sep 2019 06:40:45 GMT

Hi harsmarvania57, I try to paste the xml data here, but I'm affraid, that special chars will be removed:

  <subTest  name="subTest_name"  testPosition="unknown">
     <subPositions>
        <subPosition  name="{60}"/>
        <subPosition  name="{59}"/>
     </subPositions>
     <subTestResult  testResultClass="fail"  testResultCode="failed">
        <channel  UnitOfMeasure="V"  measureDataType="metricPrefix"  name="channel_1">
           <sample  value="17.4375m"/>
           <limit_hh  value="100m"/>
           <limit_h  value="100m"/>
           <limit_l  value="-100m"/>
           <limit_ll  value="-100m"/>
        </channel>
     </subTestResult>
  </subTest>
  <subTest  name="subTest_name"  testPosition="unknown">
     <subPositions>
        <subPosition  name="{104}"/>
        <subPosition  name="{47}"/>
     </subPositions>
     <subTestResult  testResultClass="fail"  testResultCode="failed">
        <channel  UnitOfMeasure="V"  measureDataType="decimal"  name="channel_2">
           <sample  value="1.89062"/>
           <limit_hh  value="100"/>
           <limit_h  value="100"/>
           <limit_l  value="-100"/>
           <limit_ll  value="-100"/>
        </channel>
     </subTestResult>
  </subTest>

Re: How to optimize rex to avoid the error message: Error in 'rex' command: regex= has exceeded configured match_limit, consider raising the value in limits.conf

spisiakmi — Mon, 09 Sep 2019 09:42:27 GMT

As I thought, the non complete xml code has been pasted. Please, compare it with the uploaded regex-prob.jpg file.

Re: How to optimize rex to avoid the error message: Error in 'rex' command: regex= has exceeded configured match_limit, consider raising the value in limits.conf

spisiakmi — Mon, 09 Sep 2019 10:52:38 GMT

I removed ?. The previous rex has 2568 steps. The new one has only 70 steps. But the error message still appears.
| rex "(?ms)<test\s+[^>]+^\s</test>"

Re: How to optimize rex to avoid the error message: Error in 'rex' command: regex= has exceeded configured match_limit, consider raising the value in limits.conf

harsmarvania57 — Mon, 09 Sep 2019 10:56:07 GMT

If you paste your sample data with Code Sample (button 101010) then you will able to paste special character as well.

Re: How to optimize rex to avoid the error message: Error in 'rex' command: regex= has exceeded configured match_limit, consider raising the value in limits.conf

spisiakmi — Mon, 09 Sep 2019 11:06:02 GMT

Unfortunatelly it is impossible to submit the code. Nothing happened, although I pasted the code through 101010 and tried to submit it.

Re: How to optimize rex to avoid the error message: Error in 'rex' command: regex= has exceeded configured match_limit, consider raising the value in limits.conf

spisiakmi — Mon, 09 Sep 2019 11:23:40 GMT

So I removed ? from the rex. And the steps have been reduced to 70. And the Splunk ist OK with it. No error message.
| rex "(?ms)\"<"test\s+[^>]+^\s\"<"/test>" max_match=999

Re: How to optimize rex to avoid the error message: Error in 'rex' command: regex= has exceeded configured match_limit, consider raising the value in limits.conf

spisiakmi — Mon, 09 Sep 2019 11:40:06 GMT

I also reduced the set of events: index=ind fail