best tips for speeding up searches?

[transamrit]

best tips for speeding up searches?

[puneethgowda]

We have done the following things after doing R & D.

1.Changed date range from real time to today.
2.Set dashboard refresh time to every 5 minutes.
3.Summary indexing
4.Report acceleration
5.Scheduled this search every 5 minutes so it will save in the cache.
6.Search query optimization.
7.Auto restart splunk daily at 2:00 AM UTC so that memory will be released.
8.Set high priority to this dashboard.
7.Set high priority to this scheduled search.
8.Run stats tables first then start charts.
9.Changed the delimer of raw data from text files method to new way which will reduce the time while converting raw data to fields of delimiting proccess.
10.Reduce the number of indexes and source type

After all this my dashboards loading time reduced from 3 minutes to less than 10 seconds.

Super fast

[howyagoin]

It's going to sound obvious, but, "be as specific as you can be" in your search. I've got nearly 500,000,000 events in my Splunk at the moment and I definitely get the best results for speed when I use as many of the indexed fields as possible in my query. Host, source, sourcetype, time range (important one!), index name, and so on.

As others have pointed out, if you can disable field discovery, that will help a lot as well.

[Simeon · Accepted solution]

One could write a Novel on this, but I'll focus on reporting type searches...

• Use the Advanced Charting view - this typically speeds up most searches of this type
• From a the "search" view: remove any unnecessary field extractions by: turning off field discovery; using the fields command so that it only returns the field you desire (eg - "my error | fields host").
• For reports that need to analyze millions of events AND they are run consistently, use summary indexing
• Create a dashboard that persists a saved search (see below)
• For dashboards, create a saved search and force the dashboard to use the persisted result (useHistory parameter)
• If you have multiple indexes, search the specific index you need to report on.
  
• If your result set contains indexed fields, leverage them in your search (by default Splunk indexes host source and sourcetype)
• Create a distributed search environment and leverage the map/reduce feature (add an indexer)
• For extremely large summary search reports on systems where you have many cpus available for searching, schedule parallel searches on subsets of the data
• If your search environment is distributed and you DON'T need to run it remotely, use the 'localop' command (e.g. - a local summary index search or geoip lookup)

-

[transamrit]

thanks! . .

[gnovak]

-Turn field discovery off if you haven't used any additional fields perhaps....
-select a smaller time range then "All Time"
-Perhaps your search is too generic? Try narrowing the search down to more specific data that you are looking for....

Thinking of other ways......hmmm....