topic Re: How to use OR in regex to capture error messages from two different patterns of log files? in Splunk Search

How to use OR in regex to capture error messages from two different patterns of log files?

vineetc — Wed, 07 Oct 2015 22:21:35 GMT

So I have the following log structure:

Oct  7 13:51:05, 10.96.3.29, 10.96.3.29, domain:,default [xyz][0x80e003aa][xmlparse][error] mpgw(AbcService): trans(2389130247)[request]: mismatched tag, expected meta at offset 675 of http://xyz/sales/zyx
Oct  7 13:51:05, 10.96.3.28, 10.96.3.28, domain:,default [abc][0x80e00161][mpgw][error] source-http(XyzService): trans(363541717)[x.x.x.x]: Request processing failed: Connection terminated before request headers read because of the connection error occurs, from URL: x.x.x.x:60510

I am trying to get the error_msg in a variable, but since the pattern is not distinct, I have to OR the matching pattern which matches anything after [request]: as Error and anything after [X.X.X.X]: as Error.

I created a regex which work well on a regex tester, but Splunk gives an error:

Error in 'rex' command: Encountered the following error while compiling the regex 
'\[error\][^\(]+\((?<service>[^\)]+).+(trans\((\d+){8}\)\[\b(?:\d{1,3}\.){3}\d{1,3}\b\]\:\s+(?<error_msg>.+) | trans\((\d+){8}\)\[request\]\:\s+(?<error_msg>.+))': Regex: two named subpatterns have the same name

Below is my regex:

\[error\][^\(]+\((?<service>[^\)]+).+(trans\((\d+){8}\)\[\b(?:\d{1,3}\.){3}\d{1,3}\b\]\:\s+(?<error_msg>.+) | trans\((\d+){8}\)\[request\]\:\s+(?<error_msg>.+))

How to capture the ErrorMessage from the two pattern of log files ?
Extracting the fields doesn't work either and gives a lot of junk, so I would like to go through the regex way.

Re: How to use OR in regex to capture error messages from two different patterns of log files?

masonmorales — Wed, 07 Oct 2015 22:47:28 GMT

There is a way to make your regex more efficient/cleaner but the quickest answer is:

 \[error\][^\(]+\((?<service>[^\)]+).+(trans\((\d+){8}\)\[\b(?:\d{1,3}\.){3}\d{1,3}\b\]\:\s+|trans\((\d+){8}\)\[request\]\:\s+)(?<error_msg>.+)

Use a pipe (|) within a capture group as an OR operator. You only specify (?.+) once.

Re: How to use OR in regex to capture error messages from two different patterns of log files?

woodcock — Wed, 07 Oct 2015 22:50:36 GMT

Like this:

 Your Base Search | rex  "\[error\][^\(]+\((?<service>[^\)]+).+(trans\((\d+){8}\)\[\b(?:\d{1,3}\.){3}\d{1,3}\b\]\:\s+(?<error_msg1>.+) | trans\((\d+){8}\)\[request\]\:\s+(?<error_msg2>.+))" | eval error_msg = coalesce(error_msg1, error_msg2)

Re: How to use OR in regex to capture error messages from two different patterns of log files?

masonmorales — Wed, 07 Oct 2015 22:52:09 GMT

I have not tested this but just eye balling it, looks like you could get the error message with something like:

trans\(\d+\)\[[^\]+\]\:\s+(?<error_msg>.+)

Re: How to use OR in regex to capture error messages from two different patterns of log files?

vineetc — Wed, 07 Oct 2015 22:55:29 GMT

I was able to add a character(?J) before the variable but seems like I still do get a lot of NULL in error_msg so something is there is my log which still doesnt match

Re: How to use OR in regex to capture error messages from two different patterns of log files?

vincenteous — Thu, 08 Oct 2015 02:55:50 GMT

Hi vineetc,

Try this regex pattern:

\[error\].+?:\strans\(\d+\)\[(?:request|\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\]:\s(?<error_msg>.+)

Re: How to use OR in regex to capture error messages from two different patterns of log files?

masonmorales — Thu, 08 Oct 2015 14:18:27 GMT

What about:

\]\:\s(?<error_msg>.+)