topic Re: forwarder with regex in Dashboards & Visualizations

forwarder with regex

simo — Thu, 03 Feb 2022 22:02:07 GMT

Hi All,

my solution foresees a heavy forwarder that sends data to an indexer, in the transforms.conf file I have a regex, which allows me to filter through string only the lines I need

regex example
REGEX = ^.*(?:SIMONE|MARCO).*

I am acknowledging these errors

ERROR Regex - Failed in pcre_exec: Error PCRE_ERROR_MATCHLIMIT for regex:
WARN regexExtractionProcessor - Regular expression for stanza xxxxx exceeded configured PCRE match limit. One or more fields might not have their values extracted, which can lead to incorrect search results. Fix the regular expression to improve search performance or increase the MATCH_LIMIT in props.conf to include the missing field extractions.

I wanted to know if there is a different way to filter the data to send without regex?

Best Regards,
Simone

Re: forwarder with regex

PickleRick — Thu, 03 Feb 2022 22:10:04 GMT

Are you sure your data is properly broken into events?

It looks as if you were processing a huge chunk of data at once thus hitting too many matches.

Re: forwarder with regex

ITWhisperer — Thu, 03 Feb 2022 22:59:07 GMT

Try your regex without the greedy .* since this is implied i.e. the expression will be matched anywhere in the event

REGEX = (?:SIMONE|MARCO)

Re: forwarder with regex

simo — Fri, 04 Feb 2022 15:37:17 GMT

Hi @ITWhisperer,

Thanks for the reply, I would like to remove the regex to filter. it's possible?

in the regex it is also present

REGEX = (?:SIMONE|MARCO|SIMONE.*HOME)

Simone

Re: forwarder with regex

ITWhisperer — Fri, 04 Feb 2022 15:44:03 GMT

Re: forwarder with regex

simo — Fri, 04 Feb 2022 15:55:40 GMT

sorry my example is incorrect

REGEX = (?:SIMONE|MARCO|DANIELE.*HOME)

Re: forwarder with regex

ITWhisperer — Fri, 04 Feb 2022 15:58:30 GMT

That is the correct expression to match the bold events

Re: forwarder with regex

simo — Fri, 04 Feb 2022 16:11:10 GMT

ok, but it is possible to filter without regex?

Re: forwarder with regex

ITWhisperer — Fri, 04 Feb 2022 16:18:19 GMT

Yes, you just make the REGEX match anything

REGEX = .