Multiple regex in a field extraction

[byu168]

Hi,

What I mean is that I want to parse all the error messages in my logs into one field called Errors but the regular expressions are different.

Examples:
Error: exceed max iterations, iter 120, count_trial 120
ERROR setup_acap_venv.sh failed.
ERROR [ac_analysis.tools.merge_annotations:327]

They don't quite all match up so one field extraction won't encompass all of them. Is there a way to have multiple regex that go into one field? Or is there a way to handle this when indexing the data instead of creating a field extraction?

[woodcock]

Yes, you can do this in the CLI by piping to a series of regex commands back-to-back with the same capture name. The last successful one will win but none of the unsuccessful ones will damage a previously successful field value creation.

[jwalbert]

One field extract should work, especially if your logs all lead with 'error' string prefix. Simple extraction based on your sample events:

(?i)error[\s:]+(?.*) OR (?i)error[^\w]+(?.*(?\]|\.))

The first one being the more simple/straightforward of the two, with the latter aiming to clean up the extracted data if you are so inclined.

perl -ne 'print $1.$/ if /error[^\w]+(.*(?<!\]|\.))/i' re_sample
exceed max iterations, iter 120, count_trial 120
setup_acap_venv.sh failed

ac_analysis.tools.merge_annotations:327

Joe

[micahkemp · Accepted solution]

Yes, you can definitely have multiple field extractions in to the same field.

props.conf

[<sourcetype>]
REPORT-yourfield = yourfield1,yourfield2,yourfield3

transforms.conf

[yourfield1]
REGEX = (?<yourfield>blahblahblah)

[yourfield2]
REGEX = (?<yourfield>moreblahmoreblah)

[yourfield3]
REGEX = (?<yourfield>evenmoreblah)

[byu168]

Thanks! Worked perfectly