Archive

Help with REX

Path Finder
rex "(?i)\(ms\):(?P<duration>.+)"  

Query:

sourcetype="mylog" | rex "(?i)\(ms\):(?P<duration>.+)" |  eval epochtime=_time |   sort epochtime | table epochtime,duration

I am unable to figure out what is happening within the rex statement above.

(ms):5.276144

(Data in log file comes in this format).
I can understand that we're trying to catch "(ms):" in the rex command through "\(ms\):"
but am not able to figure out the rest of it.
Kindly help.

This command is extremely slow in performance - would request what changes could be done to better the performance.

Tags (1)
0 Karma

Influencer
"(?i)\(ms\):(?P<duration>.+)"

(?i)             = case insensitive  
\(ms\):          = match (ms): literally
(?P<duration>.+) = match 1 or more characters and make this available in the 'duration' field

It is extremely unlikely that the rex is the bottleneck. For something that simple, the regex engine would process tens of thousands of lines a second, and probably a lot more.

Put your search in the search bar.
Let it run, then press the Job Inspector button (the blue one with "i" on it).

You'll see what parts of the search take longest.

command.rex will be listed somewhere - its likely that its a very short bar.

Path Finder

The following are the timings shown - looks like command.search also is taking time. There are 26,128 matching events.

1.249 command.search
0.525 command.search.typer

0.436 command.search.rawdata
0.355 command.sort
1.518 dispatch.fetch 13 - -
2.799 dispatch.preview
1.514 dispatch.stream.local
1.779 dispatch.timeline

1.173 dispatch.tmpevents
0.546 startup.handoff

0 Karma

Path Finder

To clarify - query was written by someone else who left the firm 😞
1. I didnt understand what the query is doing - I'm just getting the output of time and duration (how duration is extracted and what does + symbol denote - is it adding anything/ and overall what is the segment right to rex doing. Unable to get it.
2. Whole query is slow - assumption is that rex and segment next to rex are taking too much time by splunk to evalute.

0 Karma

Legend

So what you're saying is you wrote a query that you don't understand what it does? Is something not working, or is it working but you don't understand why?

Also I'm not aware of any particular performance issues with rex - do you mean that the performance of rex itself is poor, or do you mean that the whole query with or without rex is slow?

0 Karma