topic Re: Optimize rex command in Splunk Search

Optimize rex command

clementros — Wed, 30 Sep 2020 03:24:06 GMT

Hi all,

I want to extract fields form log events.

I have two errors patterns :

* Can not convert FOO from here ==> *
Can't find the FOO segment

What i want is to create a field named grammar_error_reason that contain "Can not convert FOO" or "Can't find the FOO segment"

Here is my rex command :

| rex field=CDREASON "Converted segments:(\s\w{3}+\s-)+\s(?<grammar_error_reason>.+) from here"

This command raise le limits of limits.conf. It take 236 steps.
In the limits.conf file the rex limits are :

[rex]
match_limit = 100000
depth_limit = 1000

Can you help me please.

Kind regards,
Clement

Re: Optimize rex command

to4kawa — Wed, 18 Dec 2019 11:53:03 GMT

| makeresults 
| eval CDREASON=" * Can not convert FOO from here ==> *
 Can't find the FOO segment"
| rex field=CDREASON max_match=100 "(Can not convert|Can't find the) (?<grammar_error_reason>\w+) (from here|segment)"

HI, @clementros
How many lines per event?
If there are more than 100 lines, increase max_match values.

Re: Optimize rex command

woodcock — Wed, 18 Dec 2019 15:32:33 GMT

Your problem has nothing to do with limits; your problem is that your RegEx is wrong. Try this:

... | rex field=CDREASON "(?<grammar_error_reason>(?:Can not convert|Can't find the)\s+\w+)\s+(?:from here|segment)"