Splunk Search

Which command or stanza can be used to decide which fields are extracted at search time to improve performance?

dannyzen
Explorer

As far as I know, fields- does not improve performance, and I'm looking for a better option.

0 Karma

DalJeanis
Legend

Improve performance on what?

If you put fields at the very top of your query, it saves a lot of extraction costs. But, generally, you want to use the positive version - tell the system the list of fields that you actually DO need, rather than the ones you don't.

Lower down, | fields - will reduce the overhead marginally, by reducing what gets passed through the following pipeline. This can be a major reduction if everything above it is a streaming command, so you save yourself from passing data from the indexers to the search head.


There are a large number of optimization techniques that are data-dependent. In my experience, most effective refactoring efforts consist of converting the query to a different search model that is more appropriate to the data mix.

If you post the individual queries as separate questions - "how can I optimize this search?" - then we can help you figure out what would work for each one.

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

For ad-hoc searches, make sure to set the search mode to 'Fast' in the UI and Splunk will skip field extraction as much as possible. For saved searches reports, 'Smart' mode is the default.

You can observe the performance difference in job inspector by looking for the command.search.kv metric.

There are many more aspects of SPL and your Splunk infrastructure itself that affect Splunk performance, so if you have a specific performance issue, please post your search and the contents of the job inspector window if you are looking for more detailed help.

0 Karma

dannyzen
Explorer

Thank you, for an ad-hoc search I just want an alternative to fields- if there is one?

0 Karma

gjanders
SplunkTrust
SplunkTrust

What is the purpose / what are you trying to achieve here?

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

Not to my knowledge, outside of setting the search mode.

0 Karma

diogofgm
SplunkTrust
SplunkTrust

The field extractions are defined in the props.conf and transforms.conf files. if you are in smart or verbose mode splunk will do all extractions that apply to your data (e.g. that apply to the sourcetypes you searching). You can build your own props/transforms to extract only the fields you need.
Nevertheless can you elaborate on the performance problem you are facing?

------------
Hope I was able to help you. If so, some karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...