Splunk Search

How to regex a log error?

Abass42
Path Finder

So I have an issue that I cant quite figure out the proper syntax for. Im parsing logs for an ERROR message. Using Search ERROR works fine, but i was wanting to regex the actual error code. Im creating my own regex, since some of the logs that come in contain different amount of lines, the usual regex cant pick up on all of them. 

Some of the example logs are 

 

 

Sat Feb 25 2023 15:11:04 ERROR:   Could not obtain time sample from 10.11.111.111 (10.11.111.111:123) using NTP (unauthenticated); error 10060: Timed out Sat Feb 25 2023 15:11:04 Info:    No server provided a usable time sample; discovering time servers to use

 

 

 

Wed Jan 18 2023 15:27:32 ERROR: Could not obtain time sample from 10.11.111.111 (10.11.111.111:123) using NTP (unauthenticated); error 10054: Connection reset Wed Jan 18 2023 15:27:32 Info: Summary: 1 sample; delta is -0.0005745 seconds, latency +0.0013496 Wed Jan 18 2023 15:27:32 Info: Alignment of -0.0005745 seconds in progress; +0.999738 secs/second (156209/156250) for 1996 ms using default method; net change -0.0005748 secs Wed Jan 18 2023 15:27:34 Info: Local clock aligned backward to match 10.82.39.229; delta -0.0005745 seconds, protocol NTP, latency 0.0013496 seconds Wed Jan 18 2023 15:27:34 Info: Next time check due in 30 seconds (fixed schedule)

 

My regex works and finds what I need, from the start of ERROR to right before the next day of the week. The regex that I used in regExr and worked on those example logs is:

 

 

(?<ERROR>(ERROR:).*(?= Sun| Mon| Tue| Wed| Thu| Fri| Sat))

 

I was also using the nongreedy flag on regexr.  But I was told that Splunk was already nongreedy. 

My issue is that when I use the rex command with that, it won't pick it up. I used a shorter iteration of that to just get and store the ERROR part, and it did grab and store the string as an ERROR field. I was following along with this guide:
http://karunsubramanian.com/splunk/how-to-use-rex-command-to-extract-fields-in-splunk/

I was just wondering how to properly implement the above working regex in Splunk syntax to grab and store the error code and short description in the field ERROR. Thank you for any guidance. 

Labels (2)
Tags (2)
0 Karma
1 Solution

yuanliu
SplunkTrust
SplunkTrust

I think you meant

 

| rex "(?<ERROR>(ERROR:).*?)\s+(?=Sun|Mon| Tue|Wed|Thu|Fri|Sat)\s"

 

to be non-aggressive.   But really it looks like you have a linebreak problem.  Using regex to work around bad data is at best a stopgap measure.  Or is this meant to be a line breaker in props.conf?  It is better to work with your developers and sysadmins to prevent logs from sticking into one line in the first place.

View solution in original post

yuanliu
SplunkTrust
SplunkTrust

I think you meant

 

| rex "(?<ERROR>(ERROR:).*?)\s+(?=Sun|Mon| Tue|Wed|Thu|Fri|Sat)\s"

 

to be non-aggressive.   But really it looks like you have a linebreak problem.  Using regex to work around bad data is at best a stopgap measure.  Or is this meant to be a line breaker in props.conf?  It is better to work with your developers and sysadmins to prevent logs from sticking into one line in the first place.

Abass42
Path Finder

So that regex does seem to work on more of the results.  Looking at the logs source, they are single line, and thats how i was testing the logs in regexr. i would delete the new space so they would all be on one line, just wrapped. I was trying to extract the error code, and i am getting about half of the results. The non match results look almost identical to the match ones. Regex_Match.pngRegex-NonMatch_Blur.png


 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

I tested both logs, one of the matches and the other non match in regexr, and its not picking the non match either. They both have the same format. They look identical except for the ips and time stamps. Im more confused than I was earlier, but your regex did at least match soem of them, so i appreciate that. Thank you. 

Im making all of the logs single line in regexr. 

 

 

0 Karma

Abass42
Path Finder

This suggested regex had a space right before tuesday, throwing it off. 

(?<ERROR>(ERROR:).*?)\s+(?=Sun|Mon|Tue|Wed|Thu|Fri|Sat)

It works for a lot more now. Thank you. This is a starting block for the final product. Thank you for your help.

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...