Splunk Search

Which command or stanza can be used to decide which fields are extracted at search time to improve performance?

dannyzen
Explorer

As far as I know, fields- does not improve performance, and I'm looking for a better option.

0 Karma

DalJeanis
Legend

Improve performance on what?

If you put fields at the very top of your query, it saves a lot of extraction costs. But, generally, you want to use the positive version - tell the system the list of fields that you actually DO need, rather than the ones you don't.

Lower down, | fields - will reduce the overhead marginally, by reducing what gets passed through the following pipeline. This can be a major reduction if everything above it is a streaming command, so you save yourself from passing data from the indexers to the search head.


There are a large number of optimization techniques that are data-dependent. In my experience, most effective refactoring efforts consist of converting the query to a different search model that is more appropriate to the data mix.

If you post the individual queries as separate questions - "how can I optimize this search?" - then we can help you figure out what would work for each one.

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

For ad-hoc searches, make sure to set the search mode to 'Fast' in the UI and Splunk will skip field extraction as much as possible. For saved searches reports, 'Smart' mode is the default.

You can observe the performance difference in job inspector by looking for the command.search.kv metric.

There are many more aspects of SPL and your Splunk infrastructure itself that affect Splunk performance, so if you have a specific performance issue, please post your search and the contents of the job inspector window if you are looking for more detailed help.

0 Karma

dannyzen
Explorer

Thank you, for an ad-hoc search I just want an alternative to fields- if there is one?

0 Karma

gjanders
SplunkTrust
SplunkTrust

What is the purpose / what are you trying to achieve here?

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

Not to my knowledge, outside of setting the search mode.

0 Karma

diogofgm
SplunkTrust
SplunkTrust

The field extractions are defined in the props.conf and transforms.conf files. if you are in smart or verbose mode splunk will do all extractions that apply to your data (e.g. that apply to the sourcetypes you searching). You can build your own props/transforms to extract only the fields you need.
Nevertheless can you elaborate on the performance problem you are facing?

------------
Hope I was able to help you. If so, some karma would be appreciated.
0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Take Action Automatically on Splunk Alerts with Red Hat Ansible Automation Platform

 Are you ready to revolutionize your IT operations? As digital transformation accelerates, the demand for ...

Calling All Security Pros: Ready to Race Through Boston?

Hey Splunkers, .conf25 is heading to Boston and we’re kicking things off with something bold, competitive, and ...

Beyond Detection: How Splunk and Cisco Integrated Security Platforms Transform ...

Financial services organizations face an impossible equation: maintain 99.9% uptime for mission-critical ...