Splunk Search

How do index and search time field extractions differ and which is better for search performance?

splunker12er
Motivator

Index time field extraction & Search Time field extraction

How do both differ ? Which has less performance impact of search query ?

chimell
Motivator

Hi splunker12er

At index time:

Index-time processes take place just before event data is actually indexed.

The following processes occur during (or before) index time:

. Default field extraction (such as host, source, sourcetype, and timestamp)
. Static or dynamic host assignment for specific inputs

. Default host assignment overrides

. Source type customization

.Index-time field extraction
. Event timestamping
. Event linebreaking
.Event segmentation (also happens at search time)

At search time:

Search-time processes take place while a search is run, as events are collected
by the search. The following processes occur at search time:

. Event segmentation (also happens at index time)

. Event type matching

. Search-time field extraction (automatic and custom field extractions,
including multivalue fields and calculated fields)
. Field aliasing
. Addition of fields from lookups

. Source type renaming
. Tagging

for more information about Index time field extraction & Search Time field extraction see the link as strive has give :

http://docs.splunk.com/Documentation/Splunk/6.1.3/Indexer/Indextimeversussearchtime
http://docs.splunk.com/Documentation/Splunk/6.1.3/Data/Configureindex-timefieldextraction

0 Karma

strive
Influencer

Splunk says:

Caution: We do not recommend that you add custom fields to the set of default fields that Splunk automatically extracts and indexes at index time, such as timestamp, punct, host, source, and sourcetype. Adding to this list of fields can negatively impact indexing performance and search times, because each indexed field increases the size of the searchable index. Indexed fields are also less flexible--whenever you make changes to your set of fields, you must re-index your entire dataset. For more information, see "Index time versus search time" in the Managing Indexers and Clusters manual.

For more details read these:

http://docs.splunk.com/Documentation/Splunk/6.1.3/Indexer/Indextimeversussearchtime

http://docs.splunk.com/Documentation/Splunk/6.1.3/Data/Configureindex-timefieldextraction

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...