search-time vs index-time field extraction

[Dan]

When would I ever consider extracting a field at index time?

[rayfoo]

Almost never. Only field that is of real importance to be derived/retrieved at index time is the _time field. The rest are usually better off being done dynamically (aka at search time)

[Lowell]

Also see some related discussion (along with some additional use cases for indexed fields) here:

• Do search-time fields have performance considerations?

[inventsekar]

the link listed on the above post returns page not found, .. (answers to community conversion issue.. )

but, actually, the page is, i thought to update this here for future readers:

https://community.splunk.com/t5/Splunk-Search/Do-search-time-fields-have-performance-considerations/...

[Dan · Accepted solution]

We do not recommend that you add custom fields to the set of default fields that Splunk automatically extracts and indexes at index time, such as timestamp, punct, host, source, and sourcetype. Adding to this list of fields can negatively impact indexing performance and search times, because each indexed field increases the size of the searchable index. Indexed fields are also less flexible--whenever you make changes to your set of fields, you must re-index your entire dataset. For more information, see "Index time versus search time" in the Admin manual.

With those caveats, there are times when you may find a need to change or add to your indexed fields. For example, you may have situations where certain search-time field extractions are noticeably impacting search performance. This can happen, for example, if you commonly search a large event set with expressions like foo!=bar or NOT foo=bar, and the field foo nearly always takes on the value bar.

Conversely, you may want to add an indexed field if the value of a search-time extracted field exists outside of the field more often than not. For example, if you commonly search only for foo=1, but 1 occurs in many events that do not have foo=1, you may want to add foo to the list of fields extracted by Splunk at index time.

In general, you should try to extract your fields at search time. For more information see "Create search-time field extractions" in the Knowledge Manager manual.

[ from http://www.splunk.com/base/Documentation/latest/Admin/Configureindex-timefieldextraction ]

[Lowell]

I'm really wondering if Splunk has changed their stance on this at all since 2010. I base this on two thoughts, (1) structured data extractions introduced in Splunk 6 uses indexed fields automatically (although the risk of "getting it wrong" is much lower in those types of cases), and (2) it's now possible to take full advantage of indexed fields using tstats, so there's significant performance gain possibilities when adding your own indexed fields.