Monitoring Splunk
Highlighted

Can we be alerted when a field extraction fails because of depth_limit ?

Hello everyone,

We have configured some automatic field extractions using regular expressions on some logs that can get really big. These field extractions are very important, if they fail we are missing critical information in our daily monitoring. At some point the field extraction didn't work and we realized that was because of the regex depth limit, when we ran it manually with rex and we got

 

 

Streamed search execute failed because: Error in 'rex' command: regex="<the_regex>" has exceeded the configured depth_limit, consider raising the value in limits.conf.

 

 

We fixed it by optimizing the regex and now it's working fine.

But we cannot be sure that the issue has been absolutely fixed, it could potentially happen again in the future. We would like to configure a Splunk alert that warns us when this type of error occurs. Does Splunk log anything about this type of error ? We could not find anything in _internal or anywhere else, or maybe we didn't look correctly ?

Thank you for your help !

0 Karma