How do index and search time field extractions differ and which is better for search performance?

[splunker12er]

Index time field extraction & Search Time field extraction

How do both differ ? Which has less performance impact of search query ?

[chimell]

Hi splunker12er

At index time:

Index-time processes take place just before event data is actually indexed.

The following processes occur during (or before) index time:

. Default field extraction (such as host, source, sourcetype, and timestamp)
. Static or dynamic host assignment for specific inputs

. Default host assignment overrides

. Source type customization

.Index-time field extraction
. Event timestamping
. Event linebreaking
.Event segmentation (also happens at search time)

At search time:

Search-time processes take place while a search is run, as events are collected
by the search. The following processes occur at search time:

. Event segmentation (also happens at index time)

. Event type matching

. Search-time field extraction (automatic and custom field extractions,
including multivalue fields and calculated fields)
. Field aliasing
. Addition of fields from lookups

. Source type renaming
. Tagging

for more information about Index time field extraction & Search Time field extraction see the link as strive has give :

http://docs.splunk.com/Documentation/Splunk/6.1.3/Indexer/Indextimeversussearchtime
http://docs.splunk.com/Documentation/Splunk/6.1.3/Data/Configureindex-timefieldextraction

[strive]

Splunk says:

Caution: We do not recommend that you add custom fields to the set of default fields that Splunk automatically extracts and indexes at index time, such as timestamp, punct, host, source, and sourcetype. Adding to this list of fields can negatively impact indexing performance and search times, because each indexed field increases the size of the searchable index. Indexed fields are also less flexible--whenever you make changes to your set of fields, you must re-index your entire dataset. For more information, see "Index time versus search time" in the Managing Indexers and Clusters manual.

For more details read these:

http://docs.splunk.com/Documentation/Splunk/6.1.3/Indexer/Indextimeversussearchtime

http://docs.splunk.com/Documentation/Splunk/6.1.3/Data/Configureindex-timefieldextraction