Splunk Search

Multiple regex in a field extraction

byu168
Path Finder

Hi,

What I mean is that I want to parse all the error messages in my logs into one field called Errors but the regular expressions are different.

Examples:
Error: exceed max iterations, iter 120, count_trial 120
ERROR setup_acap_venv.sh failed.
ERROR [ac_analysis.tools.merge_annotations:327]

They don't quite all match up so one field extraction won't encompass all of them. Is there a way to have multiple regex that go into one field? Or is there a way to handle this when indexing the data instead of creating a field extraction?

0 Karma
1 Solution

micahkemp
Champion

Yes, you can definitely have multiple field extractions in to the same field.

props.conf

[<sourcetype>]
REPORT-yourfield = yourfield1,yourfield2,yourfield3

transforms.conf

[yourfield1]
REGEX = (?<yourfield>blahblahblah)

[yourfield2]
REGEX = (?<yourfield>moreblahmoreblah)

[yourfield3]
REGEX = (?<yourfield>evenmoreblah)

View solution in original post

woodcock
Esteemed Legend

Yes, you can do this in the CLI by piping to a series of regex commands back-to-back with the same capture name. The last successful one will win but none of the unsuccessful ones will damage a previously successful field value creation.

0 Karma

jwalbert
New Member

One field extract should work, especially if your logs all lead with 'error' string prefix. Simple extraction based on your sample events:

(?i)error[\s:]+(?.*) OR (?i)error[^\w]+(?.*(?\]|\.))

The first one being the more simple/straightforward of the two, with the latter aiming to clean up the extracted data if you are so inclined.

perl -ne 'print $1.$/ if /error[^\w]+(.*(?<!\]|\.))/i' re_sample
exceed max iterations, iter 120, count_trial 120
setup_acap_venv.sh failed

ac_analysis.tools.merge_annotations:327

Joe

0 Karma

micahkemp
Champion

Yes, you can definitely have multiple field extractions in to the same field.

props.conf

[<sourcetype>]
REPORT-yourfield = yourfield1,yourfield2,yourfield3

transforms.conf

[yourfield1]
REGEX = (?<yourfield>blahblahblah)

[yourfield2]
REGEX = (?<yourfield>moreblahmoreblah)

[yourfield3]
REGEX = (?<yourfield>evenmoreblah)

byu168
Path Finder

Thanks! Worked perfectly

0 Karma
Get Updates on the Splunk Community!

Aligning Observability Costs with Business Value: Practical Strategies

 Join us for an engaging Tech Talk on Aligning Observability Costs with Business Value: Practical ...

Mastering Data Pipelines: Unlocking Value with Splunk

 In today's AI-driven world, organizations must balance the challenges of managing the explosion of data with ...

Splunk Up Your Game: Why It's Time to Embrace Python 3.9+ and OpenSSL 3.0

Did you know that for Splunk Enterprise 9.4, Python 3.9 is the default interpreter? This shift is not just a ...