Why is Dashboard Performance So Bad?

[David]

I've got a few lengthy dashboard load times that I'm trying to improve. Short of professional services, I'd like to poll the Answers audience for performance improvement suggestions. Is a 30+ second load time just the way it has to be when using a lot of charts?

General Guidelines

Things I have implemented / am implementing / am looking at so far:

• Summary Indexing
• Using One Search for Multiple Charts
• Inspecting the actual searches to analyze performance via the Job Inspector
• Run Searches in Advance and Use Cached Results
• Evaluate Server Performance with respect to Splunk
• Configure your flashtimeline appropriately
• Pull out any remaining cruft from SimpleXML to Advanced XML Conversion (Particularly EnablePreview, if not needed)

My Specific Test Case

I have a particular view (just one example from many with this issue) that looks at a small amount of data, then displays 30+ FlashCharts with different views. It's process is:

1. Set up Chrome, Time Picker
2. Run a generic search for * | head 1 to support custom binning and indexes. (why?)
3. Execute Custom Javascript to support custom binning and indexes.
4. Execute primary search, which takes 1 second to complete based on job inspector.
5. Execute 36 PostProcesses to support flashcharts.
6. Execute 36 additional PostProcesses to support textual data (sideview HTML modules are faster than SimpleResultsTable modules)

The behavior seen by the end user is as follows:

1. Time=0 - User Clicks Refresh on dashboard
2. Time=4 - Display clears of old data, we see the Loading...
3. Time=12 - All the textual data displays (Sideview HTML Modules)
4. Time=12 - All Flash on every instance of my browsers stops functioning (including my flash-based stopwatch)
5. Time=30 - All the flash charts fill in.

That is in Chrome. In Firefox, the textual data (#3) fills in at Time=31, just after the flash charts.

This search traverses about 5k of uncompressed source data (1.2k events). The server itself has two dual core Intels (so it's a couple of years old), which is generally bored and only spikes the CPU of one core for about 2 seconds during the search. The test box is brand new (nothing fancy or unusual installed) with a Core i5 2.53 GHz processor, discrete graphics and 4 GB of ram.

Is there anything else I can do to make this better?