The GUI is sooooo slow. If you run a search results are pulled in fast, the page is already loaded. The problem is when navigating around the GUI loading new pages, dashboards or even configuration settings. There is a significant delay in the server sending the html page back to the user.
Our old search head which is still online 6.3.2 is blazin fast. I'm not talking about searches I'm talking about page loads. Purely the hosting of the GUI.
We built a new search head cluster on new systems which are physical, SSD loaded, tons of RAM awesomeness.
Currently these systems are 6.5.0.
Even our stand alone non-clustered 6.5.0 search heads are slow.
Does anyone have an idea to this issue?
Does anyone have an idea where the web-hosting component of splunk is and how to adjust performance settings.
Else is there a way to decouple the hosting of the splunk GUI and move it to Apache etc?
Any chance to get bit more information
- which Browser you use?
- Can you please provide an output of Job Inspector (dispatch.fetch , dispatch.stream.remote, command.search.rawdata)
Version 64.0.3282.186 (Official Build) (64-bit)
Nothing is blocked. The waterfall is clean, so to speak, everything loads.
No Job Inspector since this is the initial load of the Search app, i.e. no search has even run. Which is why, I can see that the issue is sitting on the server (search head) side. The time to first byte is how long the browser is waiting for Splunk (splunkweb process) to process the and sent back the response.
So the question is, from a web framework point of view, what does Splunk do on the initial load. It would seem that there's zero caching of UI elements and it's generating everything each time.
Yes. In fact, it was immediately better. Once it went out with the deploy, the initial response, TTFB, dropped from 10+ seconds to between 1-4.
In our UI cluster, we added the following to web.conf.
max_view_cache_size = 2000
I'm seeing similarly poor performance as well in the UI. The TTFB is very slow in the UI cluster. ~14s until First Byte.
We have theory that it may be due to the sheer number of dashboards in the Search app versus say specific applications that have been setup in the UI. But it's been challenging to nail down what the splunkweb process is looking to load in the initial request before it send it's first byte.