Splunk Search

Is there a performance advantage of using rex in a search versus saving it as an extracted field?

HattrickNZ
Motivator

What is the advantage of using rex in a search V saving it as an extracted field?

Example of using rex in a search:

index=core ....|  
rex ".*,.*CPU:CPU=(?P<CPU>[^,\"]+)" | 
rex "Core:Core ID.=(?P<coreID>\d*)," | 
rex "Subrack No.=(?P<subNo>\d*)" | 
rex "Slot No.=(?P<slotNo>\d*)," | 
strcat userLabel "-" coreID "-" slotNo "-" subNo  object_formatted | 
timechart useother=f span=1h avg(c117498312) by object_formatted

But I could also save it as an extracted field and tie it to some sourcetype.

Mainly interested in speed, so is one way faster than the other?
Does it just make your search shorter?

0 Karma

lguinn2
Legend

IMO the comment from @cramasta is good: "If you don't always need the extracted fields you might be better off just using the Rex when needed."

Of course, saved field extractions are convenient. It makes it much easier for users who are less familiar with the data (or regex) to search and report. However, one of the biggest expenses at search time is field extraction. Using the rex command means that the field is only extracted when it is actually used.

While @somesoni2 may have experienced a speed up with saved field extractions (I haven't), the problem with them is this: a saved field extraction is invoked on every search that retrieves this data. The "smart" and "fast" search modes mitigate this problem to a great extent, especially for reports. Still, having many field extractions can slow event searches.

A lot of the time, it is much more important to make the data useful than it is to lose a little speed. But if performance is the most important thing, I would continue to use rex.

cramasta
Builder

Rex will opererate on ALL data returned by the search. Saved Field extractions would work on only the host/source/sourcetype you assign.

So if your search is returning events from two sources and only one source needs the extraction then a field extration I imagine it would be faster. If they both need the extraction then it probably won't make a difference which one you use.

If you don't always need the extracted fields you might be better off just using the Rex when needed.

0 Karma

somesoni2
Revered Legend

Saved it as an extracted field will improve the performance, (From my experience).

Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...