Splunk Search

Optimize rex command

clementros
Path Finder

Hi all,

I want to extract fields form log events.

I have two errors patterns :

EDICPP 4-1-1-0 exception: Mandatory element is missing.  Error at <src/CBRSegmentDecoders.cpp:21405 Error in segment CSC 06 1 1A TKA in element #2. 
    ------  Error at <src/CBRMessageDecoders.cpp:121 Error in group Group id: _294728_G_CBR in element #3. 
    ------  Decoding error in message AIRRQT 15 2 1A PNR in element #4 at character 240 of the buffer, using charset B  Converted segments: ORG - BLK - AMD - AVC - BIN -  Cannot convert CSC from here==>+9998WSSU'Last segment correctly processed: CSC+7906:/++9998WSSU'

and

Can't find the UNH segment.

What i want is to create a field named grammar_error_reason that contain exactly "Can not convert FOO" or "Can't find the FOO segment".

Here is my rex command :

(Converted segments:(\s\w{3}+\s-)+\s|)(?<grammar_error_reason>.+) (from here|segment\.)

Can you help me please.

Kind regards,
Clement

0 Karma
1 Solution

clementros
Path Finder

Thank you @richgalloway,

This command works for me :

index="dbtkbkmalformed" CDCATEGORY=GRAMMAR_ERROR 
| rex "Converted segments:(\s\w{3}+\s-)+\s+(?<grammar_error_reason1>[\w\s]+) from here" 
| rex "(?<grammar_error_reason2>Can't find the \w+ segment\.)" 
| eval grammar_error_reason = coalesce(grammar_reason1, grammar_error_reason2)

View solution in original post

0 Karma

clementros
Path Finder

Thank you @richgalloway,

This command works for me :

index="dbtkbkmalformed" CDCATEGORY=GRAMMAR_ERROR 
| rex "Converted segments:(\s\w{3}+\s-)+\s+(?<grammar_error_reason1>[\w\s]+) from here" 
| rex "(?<grammar_error_reason2>Can't find the \w+ segment\.)" 
| eval grammar_error_reason = coalesce(grammar_reason1, grammar_error_reason2)
0 Karma

woodcock
Esteemed Legend

Be sure to do something with your duplicate question:
https://answers.splunk.com/answers/790422/optimize-rex-command.html

0 Karma

richgalloway
SplunkTrust
SplunkTrust

This regex may work for you. If it doesn't, consider using two rex commands - one for each error pattern,

... | rex "(?J)(Converted segments:(\s\w{3}+\s-)+\s+(?<grammar_error_reason>.+) from here|(?<grammar_error_reason>Can't find the \w+ segment\.))"
---
If this reply helps you, Karma would be appreciated.
0 Karma

clementros
Path Finder

Hi @richgalloway,

Unfortunatly your rex command do not work cause it exceeded configured match_limit.

If i use two rex command to extract info i think i will have a problem. The regex to extract the second pattern "Can't find the UNH segment." also match with the other pattern.

How can i say if you match with the first do not try the second ?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try adding the max_match=0 option to the rex command.

Two separate rex commands should work since they each search for different strings. We can avoid conflicts using two fields and coalesce.

... | rex "(Converted segments:(\s\w{3}+\s-)+\s+(?<grammar_error_reason1>.+) from here" 
| rex "(?<grammar_error_reason2>Can't find the \w+ segment\.)"
| eval grammar_error_reason = coalesce(grammar_reason1, grammar_error_reason2)
---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Build Scalable Security While Moving to Cloud - Guide From Clayton Homes

 Clayton Homes faced the increased challenge of strengthening their security posture as they went through ...

Mission Control | Explore the latest release of Splunk Mission Control (2.3)

We’re happy to announce the release of Mission Control 2.3 which includes several new and exciting features ...

Cloud Platform | Migrating your Splunk Cloud deployment to Python 3.7

Python 2.7, the last release of Python 2, reached End of Life back on January 1, 2020. As part of our larger ...