topic Multiple regex in a field extraction in Splunk Search

Multiple regex in a field extraction

byu168 — Tue, 29 Sep 2020 14:12:15 GMT

Hi,

What I mean is that I want to parse all the error messages in my logs into one field called Errors but the regular expressions are different.

Examples:
Error: exceed max iterations, iter 120, count_trial 120
ERROR setup_acap_venv.sh failed.
ERROR [ac_analysis.tools.merge_annotations:327]

They don't quite all match up so one field extraction won't encompass all of them. Is there a way to have multiple regex that go into one field? Or is there a way to handle this when indexing the data instead of creating a field extraction?

Re: Multiple regex in a field extraction

micahkemp — Thu, 25 May 2017 18:46:15 GMT

Yes, you can definitely have multiple field extractions in to the same field.

props.conf

[<sourcetype>]
REPORT-yourfield = yourfield1,yourfield2,yourfield3

transforms.conf

[yourfield1]
REGEX = (?<yourfield>blahblahblah)

[yourfield2]
REGEX = (?<yourfield>moreblahmoreblah)

[yourfield3]
REGEX = (?<yourfield>evenmoreblah)

Re: Multiple regex in a field extraction

jwalbert — Thu, 25 May 2017 19:06:04 GMT

One field extract should work, especially if your logs all lead with 'error' string prefix. Simple extraction based on your sample events:

(?i)error[\s:]+(?.*) OR (?i)error[^\w]+(?.*(?\]|\.))

The first one being the more simple/straightforward of the two, with the latter aiming to clean up the extracted data if you are so inclined.

perl -ne 'print $1.$/ if /error[^\w]+(.*(?<!\]|\.))/i' re_sample
exceed max iterations, iter 120, count_trial 120
setup_acap_venv.sh failed

`ac_analysis.tools.merge_annotations:327`

Joe

Re: Multiple regex in a field extraction

woodcock — Thu, 25 May 2017 19:08:34 GMT

Yes, you can do this in the CLI by piping to a series of regex commands back-to-back with the same capture name. The last successful one will win but none of the unsuccessful ones will damage a previously successful field value creation.

Re: Multiple regex in a field extraction

byu168 — Thu, 25 May 2017 21:33:56 GMT

Thanks! Worked perfectly