topic Re: Which command or stanza can be used to decide which fields are extracted at search time to improve performance? in Splunk Search

Which command or stanza can be used to decide which fields are extracted at search time to improve performance?

dannyzen — Fri, 13 Oct 2017 22:47:32 GMT

As far as I know, fields- does not improve performance, and I'm looking for a better option.

Re: Which command or stanza can be used to decide which fields are extracted at search time to improve performance?

diogofgm — Fri, 13 Oct 2017 22:59:30 GMT

The field extractions are defined in the props.conf and transforms.conf files. if you are in smart or verbose mode splunk will do all extractions that apply to your data (e.g. that apply to the sourcetypes you searching). You can build your own props/transforms to extract only the fields you need.
Nevertheless can you elaborate on the performance problem you are facing?

Re: Which command or stanza can be used to decide which fields are extracted at search time to improve performance?

s2_splunk — Fri, 13 Oct 2017 23:04:12 GMT

For ad-hoc searches, make sure to set the search mode to 'Fast' in the UI and Splunk will skip field extraction as much as possible. For saved searches reports, 'Smart' mode is the default.

You can observe the performance difference in job inspector by looking for the command.search.kv metric.

There are many more aspects of SPL and your Splunk infrastructure itself that affect Splunk performance, so if you have a specific performance issue, please post your search and the contents of the job inspector window if you are looking for more detailed help.

Re: Which command or stanza can be used to decide which fields are extracted at search time to improve performance?

dannyzen — Fri, 13 Oct 2017 23:26:20 GMT

Thank you, for an ad-hoc search I just want an alternative to fields- if there is one?

Re: Which command or stanza can be used to decide which fields are extracted at search time to improve performance?

s2_splunk — Fri, 13 Oct 2017 23:29:30 GMT

Not to my knowledge, outside of setting the search mode.

Re: Which command or stanza can be used to decide which fields are extracted at search time to improve performance?

gjanders — Sat, 14 Oct 2017 02:09:42 GMT

What is the purpose / what are you trying to achieve here?

Re: Which command or stanza can be used to decide which fields are extracted at search time to improve performance?

DalJeanis — Sat, 14 Oct 2017 21:12:50 GMT

Improve performance on what?

If you put fields at the very top of your query, it saves a lot of extraction costs. But, generally, you want to use the positive version - tell the system the list of fields that you actually DO need, rather than the ones you don't.

Lower down, | fields - will reduce the overhead marginally, by reducing what gets passed through the following pipeline. This can be a major reduction if everything above it is a streaming command, so you save yourself from passing data from the indexers to the search head.

There are a large number of optimization techniques that are data-dependent. In my experience, most effective refactoring efforts consist of converting the query to a different search model that is more appropriate to the data mix.

If you post the individual queries as separate questions - "how can I optimize this search?" - then we can help you figure out what would work for each one.