Splunk Search

How do index and search time field extractions differ and which is better for search performance?

splunker12er
Motivator

Index time field extraction & Search Time field extraction

How do both differ ? Which has less performance impact of search query ?

chimell
Motivator

Hi splunker12er

At index time:

Index-time processes take place just before event data is actually indexed.

The following processes occur during (or before) index time:

. Default field extraction (such as host, source, sourcetype, and timestamp)
. Static or dynamic host assignment for specific inputs

. Default host assignment overrides

. Source type customization

.Index-time field extraction
. Event timestamping
. Event linebreaking
.Event segmentation (also happens at search time)

At search time:

Search-time processes take place while a search is run, as events are collected
by the search. The following processes occur at search time:

. Event segmentation (also happens at index time)

. Event type matching

. Search-time field extraction (automatic and custom field extractions,
including multivalue fields and calculated fields)
. Field aliasing
. Addition of fields from lookups

. Source type renaming
. Tagging

for more information about Index time field extraction & Search Time field extraction see the link as strive has give :

http://docs.splunk.com/Documentation/Splunk/6.1.3/Indexer/Indextimeversussearchtime
http://docs.splunk.com/Documentation/Splunk/6.1.3/Data/Configureindex-timefieldextraction

0 Karma

strive
Influencer

Splunk says:

Caution: We do not recommend that you add custom fields to the set of default fields that Splunk automatically extracts and indexes at index time, such as timestamp, punct, host, source, and sourcetype. Adding to this list of fields can negatively impact indexing performance and search times, because each indexed field increases the size of the searchable index. Indexed fields are also less flexible--whenever you make changes to your set of fields, you must re-index your entire dataset. For more information, see "Index time versus search time" in the Managing Indexers and Clusters manual.

For more details read these:

http://docs.splunk.com/Documentation/Splunk/6.1.3/Indexer/Indextimeversussearchtime

http://docs.splunk.com/Documentation/Splunk/6.1.3/Data/Configureindex-timefieldextraction

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...