Splunk Search

Optimize rex command

clementros
Path Finder

Hi all,

I want to extract fields form log events.

I have two errors patterns :

EDICPP 4-1-1-0 exception: Mandatory element is missing.  Error at <src/CBRSegmentDecoders.cpp:21405 Error in segment CSC 06 1 1A TKA in element #2. 
    ------  Error at <src/CBRMessageDecoders.cpp:121 Error in group Group id: _294728_G_CBR in element #3. 
    ------  Decoding error in message AIRRQT 15 2 1A PNR in element #4 at character 240 of the buffer, using charset B  Converted segments: ORG - BLK - AMD - AVC - BIN -  Cannot convert CSC from here==>+9998WSSU'Last segment correctly processed: CSC+7906:/++9998WSSU'

and

Can't find the UNH segment.

What i want is to create a field named grammar_error_reason that contain exactly "Can not convert FOO" or "Can't find the FOO segment".

Here is my rex command :

(Converted segments:(\s\w{3}+\s-)+\s|)(?<grammar_error_reason>.+) (from here|segment\.)

Can you help me please.

Kind regards,
Clement

0 Karma
1 Solution

clementros
Path Finder

Thank you @richgalloway,

This command works for me :

index="dbtkbkmalformed" CDCATEGORY=GRAMMAR_ERROR 
| rex "Converted segments:(\s\w{3}+\s-)+\s+(?<grammar_error_reason1>[\w\s]+) from here" 
| rex "(?<grammar_error_reason2>Can't find the \w+ segment\.)" 
| eval grammar_error_reason = coalesce(grammar_reason1, grammar_error_reason2)

View solution in original post

0 Karma

clementros
Path Finder

Thank you @richgalloway,

This command works for me :

index="dbtkbkmalformed" CDCATEGORY=GRAMMAR_ERROR 
| rex "Converted segments:(\s\w{3}+\s-)+\s+(?<grammar_error_reason1>[\w\s]+) from here" 
| rex "(?<grammar_error_reason2>Can't find the \w+ segment\.)" 
| eval grammar_error_reason = coalesce(grammar_reason1, grammar_error_reason2)
0 Karma

woodcock
Esteemed Legend

Be sure to do something with your duplicate question:
https://answers.splunk.com/answers/790422/optimize-rex-command.html

0 Karma

richgalloway
SplunkTrust
SplunkTrust

This regex may work for you. If it doesn't, consider using two rex commands - one for each error pattern,

... | rex "(?J)(Converted segments:(\s\w{3}+\s-)+\s+(?<grammar_error_reason>.+) from here|(?<grammar_error_reason>Can't find the \w+ segment\.))"
---
If this reply helps you, Karma would be appreciated.
0 Karma

clementros
Path Finder

Hi @richgalloway,

Unfortunatly your rex command do not work cause it exceeded configured match_limit.

If i use two rex command to extract info i think i will have a problem. The regex to extract the second pattern "Can't find the UNH segment." also match with the other pattern.

How can i say if you match with the first do not try the second ?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try adding the max_match=0 option to the rex command.

Two separate rex commands should work since they each search for different strings. We can avoid conflicts using two fields and coalesce.

... | rex "(Converted segments:(\s\w{3}+\s-)+\s+(?<grammar_error_reason1>.+) from here" 
| rex "(?<grammar_error_reason2>Can't find the \w+ segment\.)"
| eval grammar_error_reason = coalesce(grammar_reason1, grammar_error_reason2)
---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In November, the Splunk Threat Research Team had one release of new security content via the Enterprise ...

Index This | Divide 100 by half. What do you get?

November 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Stay Connected: Your Guide to December Tech Talks, Office Hours, and Webinars!

❄️ Celebrate the season with our December lineup of Community Office Hours, Tech Talks, and Webinars! ...