Splunk Search

Optimize rex command

clementros
Path Finder

Hi all,

I want to extract fields form log events.

I have two errors patterns :

EDICPP 4-1-1-0 exception: Mandatory element is missing.  Error at <src/CBRSegmentDecoders.cpp:21405 Error in segment CSC 06 1 1A TKA in element #2. 
    ------  Error at <src/CBRMessageDecoders.cpp:121 Error in group Group id: _294728_G_CBR in element #3. 
    ------  Decoding error in message AIRRQT 15 2 1A PNR in element #4 at character 240 of the buffer, using charset B  Converted segments: ORG - BLK - AMD - AVC - BIN -  Cannot convert CSC from here==>+9998WSSU'Last segment correctly processed: CSC+7906:/++9998WSSU'

and

Can't find the UNH segment.

What i want is to create a field named grammar_error_reason that contain exactly "Can not convert FOO" or "Can't find the FOO segment".

Here is my rex command :

(Converted segments:(\s\w{3}+\s-)+\s|)(?<grammar_error_reason>.+) (from here|segment\.)

Can you help me please.

Kind regards,
Clement

0 Karma
1 Solution

clementros
Path Finder

Thank you @richgalloway,

This command works for me :

index="dbtkbkmalformed" CDCATEGORY=GRAMMAR_ERROR 
| rex "Converted segments:(\s\w{3}+\s-)+\s+(?<grammar_error_reason1>[\w\s]+) from here" 
| rex "(?<grammar_error_reason2>Can't find the \w+ segment\.)" 
| eval grammar_error_reason = coalesce(grammar_reason1, grammar_error_reason2)

View solution in original post

0 Karma

clementros
Path Finder

Thank you @richgalloway,

This command works for me :

index="dbtkbkmalformed" CDCATEGORY=GRAMMAR_ERROR 
| rex "Converted segments:(\s\w{3}+\s-)+\s+(?<grammar_error_reason1>[\w\s]+) from here" 
| rex "(?<grammar_error_reason2>Can't find the \w+ segment\.)" 
| eval grammar_error_reason = coalesce(grammar_reason1, grammar_error_reason2)
0 Karma

woodcock
Esteemed Legend

Be sure to do something with your duplicate question:
https://answers.splunk.com/answers/790422/optimize-rex-command.html

0 Karma

richgalloway
SplunkTrust
SplunkTrust

This regex may work for you. If it doesn't, consider using two rex commands - one for each error pattern,

... | rex "(?J)(Converted segments:(\s\w{3}+\s-)+\s+(?<grammar_error_reason>.+) from here|(?<grammar_error_reason>Can't find the \w+ segment\.))"
---
If this reply helps you, Karma would be appreciated.
0 Karma

clementros
Path Finder

Hi @richgalloway,

Unfortunatly your rex command do not work cause it exceeded configured match_limit.

If i use two rex command to extract info i think i will have a problem. The regex to extract the second pattern "Can't find the UNH segment." also match with the other pattern.

How can i say if you match with the first do not try the second ?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try adding the max_match=0 option to the rex command.

Two separate rex commands should work since they each search for different strings. We can avoid conflicts using two fields and coalesce.

... | rex "(Converted segments:(\s\w{3}+\s-)+\s+(?<grammar_error_reason1>.+) from here" 
| rex "(?<grammar_error_reason2>Can't find the \w+ segment\.)"
| eval grammar_error_reason = coalesce(grammar_reason1, grammar_error_reason2)
---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...