Splunk Search

Is my regex or the Interactive Field Extractor regex more efficient? Is there a better option?

Path Finder

Comparing regex strings...

Log format:

Thu 08/07/2014, 6:41:59.97,USERA,TERM1,XXXX-YYYAPP65-5
Thu 08/07/2014, 6:42:00.17,USERA,,XXXX-YYYAPP65-5
Thu 08/07/2014, 6:43:55.11,USERB,TERM2,XXXX-YYYAPP65-6
Thu 08/07/2014, 6:44:25.64,USERC,TERM3,XXXX-YYYAPP65-2
Thu 08/07/2014, 6:44:58.82,USERD,TERM4,XXXX-YYYAPP65-4
Thu 08/07/2014, 6:44:58.92,USERD,,XXXX-YYYAPP65-4

alt text
(I realized the Splunk-generated regex in the image above (the Markdown was messing my regex string so I uploaded an image) was omitting extracting empty fields for cname, so cname was not defined for those rows)

Is my regex (see image above) less efficient to extract the fields?

Ultra Champion

Perhaps I'm missing something, but wouldn't this be easier to do a REPORT with DELIMS/FIELDS. It seems like a classic example.

props.conf

[my_sourcetype]
REPORT-blah = get_my_fields

transforms.conf

[get_my_fields]
DELIMS = ","
FIELDS = date, time, userid, term, blah, bleh, bluh

Makes sense?

/k

Path Finder

I've tended to stay away from .conf files, and do most of my parsing/extracting in-line in the query, but I'll give the .conf file a shot in a development instance to see if it will work for my needs. Thanks!

0 Karma

Ultra Champion

Haha, it's the answer to a different question! Perhaps the example in the original post is just an example.

Anyway, how does DELIMS work under the hood - is it based on regex? I mean, I could see that

DELIMS = ","

is easily stated as

([^,]*),\s*?

But is it another mechanism that actually does it? Could it be more efficient to write an EXTRACT or REPORT regex, than to rely on DELIMS/FIELDS?

/k

0 Karma

Legend

better answer!

0 Karma

Legend

Your regex is not less efficient - but it could be problematic. By default, .* is greedy. I would avoid that by doing this

| rex "(?<day1>.*?)\s(?<date1>.*?),(?<time1>.*?),(?<uname>.*?),(?<cname>.*?),(?<sname>.*?)"

Path Finder

Thanks for the tip on the frugal regex! 🙂

0 Karma

SplunkTrust
SplunkTrust

I'd say the regex (?<day1>.*) (?<date1>.*),... is very inefficient. Due to the greediness, the regex will at first match the entire string into day1 and backtrack its way back to the most recent space after the date. Then it'll match the rest from the time to the end into date1, and backtrack to find a comma. That process will take massively more time and memory than basically stepping linearly through the string once as @lguinn's regex would.

Non-greedy is a simple fix, alternatively you could use more accurate matches than the dot - for example, use \w or \S for day1.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!