topic Re: Efficiency of REGEX = . vs REGEX = .* vs REGEX = (.*) in Splunk Search

Efficiency of REGEX = . vs REGEX = .* vs REGEX = (.*)

chris_barrett — Wed, 30 Sep 2020 04:01:11 GMT

Which of the following (in terms of the REGEX) is the most efficient? I've seen examples of all of them. And is there an even more efficient REGEX?

[send_to_nullQueue_dot]
SOURCE_KEY = _raw
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[send_to_nullQueue_dot_star]
SOURCE_KEY = _raw
REGEX = .*
DEST_KEY = queue
FORMAT = nullQueue

[send_to_nullQueue_dot_plus]
SOURCE_KEY = _raw
REGEX = .+
DEST_KEY = queue
FORMAT = nullQueue

[send_to_nullQueue_dot_star_lazy]
SOURCE_KEY = _raw
REGEX = .*?
DEST_KEY = queue
FORMAT = nullQueue

[send_to_nullQueue_captured_dot_star]
SOURCE_KEY = _raw
REGEX = (.*)
DEST_KEY = queue
FORMAT = nullQueue

Re: Efficiency of REGEX = . vs REGEX = .* vs REGEX = (.*)

to4kawa — Mon, 03 Feb 2020 10:05:14 GMT

Matching is all different.
if it works, . is best.
but it works?

Re: Efficiency of REGEX = . vs REGEX = .* vs REGEX = (.*)

PavelP — Mon, 03 Feb 2020 10:54:08 GMT

Most efficient in regard of speed? You can test your regexes with https://regex101.com/ and compare "number of steps" shown right on top of the regex field.

REGEX = . doesn't make sense because it matches any pattern.

My bet: fixed case-sensitive pattern with static quanitifiers anchored to beginning of the line? 🙂

There are a lot of information about regexes on World Wide Web, hier is splunk focused: https://conf.splunk.com/files/2016/slides/become-a-regular-expressions-ninja-and-unlock-your-splunk-potential.pdf

Re: Efficiency of REGEX = . vs REGEX = .* vs REGEX = (.*)

DalJeanis — Mon, 03 Feb 2020 17:27:45 GMT

A single dot matches any single character that is in the input line. That's the fastest, most effective one.

All the other ones are more complicated than that, so would be slightly slower.

. any single character
.* zero or more characters but scan all of them
.+ one or more characters but scan all of them
.+? zero or more characters, scan up to all of them but be wishy washy about it
(.*) zero or more characters, scan all of them, but collect the whole thing as a field and then do nothing with it.

Re: Efficiency of REGEX = . vs REGEX = .* vs REGEX = (.*)

chris_barrett — Mon, 03 Feb 2020 20:38:16 GMT

In hindsight, I should have said that the value of _raw was irrelevant as the decision to not index the event was being made on the basis of either the host, source or sourcetype. So the transforms stanza was really just about setting the queue to nullQueue.

And thank you for the link to the Conf presentation. I had seen it before but it was good to go over it again.