Splunk Search
Highlighted

Which command or stanza can be used to decide which fields are extracted at search time to improve performance?

Explorer

As far as I know, fields- does not improve performance, and I'm looking for a better option.

0 Karma
Highlighted

Re: Which command or stanza can be used to decide which fields are extracted at search time to improve performance?

Motivator

The field extractions are defined in the props.conf and transforms.conf files. if you are in smart or verbose mode splunk will do all extractions that apply to your data (e.g. that apply to the sourcetypes you searching). You can build your own props/transforms to extract only the fields you need.
Nevertheless can you elaborate on the performance problem you are facing?

------------
Hope I was able to help you. If so, an upvote would be appreciated.
0 Karma
Highlighted

Re: Which command or stanza can be used to decide which fields are extracted at search time to improve performance?

Splunk Employee
Splunk Employee

For ad-hoc searches, make sure to set the search mode to 'Fast' in the UI and Splunk will skip field extraction as much as possible. For saved searches reports, 'Smart' mode is the default.

You can observe the performance difference in job inspector by looking for the command.search.kv metric.

There are many more aspects of SPL and your Splunk infrastructure itself that affect Splunk performance, so if you have a specific performance issue, please post your search and the contents of the job inspector window if you are looking for more detailed help.

0 Karma
Highlighted

Re: Which command or stanza can be used to decide which fields are extracted at search time to improve performance?

Explorer

Thank you, for an ad-hoc search I just want an alternative to fields- if there is one?

0 Karma
Highlighted

Re: Which command or stanza can be used to decide which fields are extracted at search time to improve performance?

Splunk Employee
Splunk Employee

Not to my knowledge, outside of setting the search mode.

0 Karma
Highlighted

Re: Which command or stanza can be used to decide which fields are extracted at search time to improve performance?

SplunkTrust
SplunkTrust

What is the purpose / what are you trying to achieve here?

0 Karma
Highlighted

Re: Which command or stanza can be used to decide which fields are extracted at search time to improve performance?

SplunkTrust
SplunkTrust

Improve performance on what?

If you put fields at the very top of your query, it saves a lot of extraction costs. But, generally, you want to use the positive version - tell the system the list of fields that you actually DO need, rather than the ones you don't.

Lower down, | fields - will reduce the overhead marginally, by reducing what gets passed through the following pipeline. This can be a major reduction if everything above it is a streaming command, so you save yourself from passing data from the indexers to the search head.


There are a large number of optimization techniques that are data-dependent. In my experience, most effective refactoring efforts consist of converting the query to a different search model that is more appropriate to the data mix.

If you post the individual queries as separate questions - "how can I optimize this search?" - then we can help you figure out what would work for each one.

0 Karma