Splunk Search

Optimize rex command

clementros
Path Finder

Hi all,

I want to extract fields form log events.

I have two errors patterns :

EDICPP 4-1-1-0 exception: Mandatory element is missing.  Error at <src/CBRSegmentDecoders.cpp:21405 Error in segment CSC 06 1 1A TKA in element #2. 
    ------  Error at <src/CBRMessageDecoders.cpp:121 Error in group Group id: _294728_G_CBR in element #3. 
    ------  Decoding error in message AIRRQT 15 2 1A PNR in element #4 at character 240 of the buffer, using charset B  Converted segments: ORG - BLK - AMD - AVC - BIN -  Cannot convert CSC from here==>+9998WSSU'Last segment correctly processed: CSC+7906:/++9998WSSU'

and

Can't find the UNH segment.

What i want is to create a field named grammar_error_reason that contain exactly "Can not convert FOO" or "Can't find the FOO segment".

Here is my rex command :

(Converted segments:(\s\w{3}+\s-)+\s|)(?<grammar_error_reason>.+) (from here|segment\.)

Can you help me please.

Kind regards,
Clement

0 Karma
1 Solution

clementros
Path Finder

Thank you @richgalloway,

This command works for me :

index="dbtkbkmalformed" CDCATEGORY=GRAMMAR_ERROR 
| rex "Converted segments:(\s\w{3}+\s-)+\s+(?<grammar_error_reason1>[\w\s]+) from here" 
| rex "(?<grammar_error_reason2>Can't find the \w+ segment\.)" 
| eval grammar_error_reason = coalesce(grammar_reason1, grammar_error_reason2)

View solution in original post

0 Karma

clementros
Path Finder

Thank you @richgalloway,

This command works for me :

index="dbtkbkmalformed" CDCATEGORY=GRAMMAR_ERROR 
| rex "Converted segments:(\s\w{3}+\s-)+\s+(?<grammar_error_reason1>[\w\s]+) from here" 
| rex "(?<grammar_error_reason2>Can't find the \w+ segment\.)" 
| eval grammar_error_reason = coalesce(grammar_reason1, grammar_error_reason2)
0 Karma

woodcock
Esteemed Legend

Be sure to do something with your duplicate question:
https://answers.splunk.com/answers/790422/optimize-rex-command.html

0 Karma

richgalloway
SplunkTrust
SplunkTrust

This regex may work for you. If it doesn't, consider using two rex commands - one for each error pattern,

... | rex "(?J)(Converted segments:(\s\w{3}+\s-)+\s+(?<grammar_error_reason>.+) from here|(?<grammar_error_reason>Can't find the \w+ segment\.))"
---
If this reply helps you, Karma would be appreciated.
0 Karma

clementros
Path Finder

Hi @richgalloway,

Unfortunatly your rex command do not work cause it exceeded configured match_limit.

If i use two rex command to extract info i think i will have a problem. The regex to extract the second pattern "Can't find the UNH segment." also match with the other pattern.

How can i say if you match with the first do not try the second ?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try adding the max_match=0 option to the rex command.

Two separate rex commands should work since they each search for different strings. We can avoid conflicts using two fields and coalesce.

... | rex "(Converted segments:(\s\w{3}+\s-)+\s+(?<grammar_error_reason1>.+) from here" 
| rex "(?<grammar_error_reason2>Can't find the \w+ segment\.)"
| eval grammar_error_reason = coalesce(grammar_reason1, grammar_error_reason2)
---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...