We are sharing multiple dashboards with clients which are automatically refreshing every 5 minutes.
The problem is that from time to time, the dashboard doesn't display panels properly as shown below
!Is there something wrong with the xml code or is it simply a performance issue? We have added new indexer which should have enhanced the performance of Splunk but we are not experiencing any difference. Is there a query to check the performance degradation before and after?
I would assume this is performace related since it only fails from time to time. Adding indexers is perhaps not the only way to deal with high performance needs due to a higher number of users, remeber that the search heads also play a role in this (assuming you have a distributed environment).
Have you had a look at your internal indexes and at individual searches to see which parts of your system are pressured most/take the longest?
I would suggest you have a look here for an overview of what you can find. Unfortunately I can't hint you directly at every possible cause, but I would imagine that it's best to have a look at the metrics.log - this log contains many important, well, "metrics" about your splunk instance 😉
Of course, checking for client-side issues is always a good idea. Switching to another browser is an easy first step to determine if the first one is the cause!
I haven't check internal indexes but I think I found the issue. We are using FireFox as our default browser and it is consuming 80% of physical memory on average. If shifting tabs or queries I type appear a minute later, it is issue with the browser and RAM, right?