Splunk lagging on user's desktops

[Ricapar]

We've built up a fairly decent Splunk infrastructure - several machines with lots of horsepower, etc. We can take on several times the load we're currently giving them.

However, something seems to have been overlooked a bit.

The majority of our users don't have the most powerful machines sitting on their desks. Some also connect through a VDI session, which is basically streaming your desktop over the network.

As we're building more data-rich dashboards, users are starting to complain that Splunk is "slow". Took us a bit to realize that by "slow" they didn't mean that the backend was taking a long time.. but that whenever they load a dashboard, their CPU is basically pegged, and you can hardly scroll up or down a page, much less hover over pieces of a graph or chart without crazy amounts of lag.

We're working on reducing the amount of items on certain dashboards to help minimize the load on user's browsers/desktops. However, that seems like a bandaid to the problem right now.

Aside of giving everyone desktop machines with more horsepower (not gonna happen), what else can we do?

I'd also like to add that we're going to be trying to force the charts/graphs to use FlashCharts over the JavaScript charts to see if it makes any difference. Can/will that make a noticeable difference?

Edit: Extra information
Edit Again: Correction - users on are on IE 8, not 9
Users are using IE, as per the corporate standard. Though there are some that have "acquired" Firefox and/or Chrome through other means. However, those aren't sanctioned by the enterprise, and the majority of users are on IE 8.

For the few that have Chrome and Firefox, there doesn't seem to be many performance issues unless someone creates a massive dashboard. It doesn't seem to take much to push IE over the edge though.

[gkanapathy]

Using Flash may make a difference, depending on your browser. You haven't said what browser you are using, but there is a significant difference between a new version of Chrome or Safari and an older version of IE.

As a very last resort, you may have to create simpler views, or in the extreme, use the Splunk REST API to create a drastically simplified UI by querying and rendering the results on the server.

[Ricapar]

@bmacias84: JSCharts was the default.. and seems to be what IE takes the biggest hit on. FlashCharts seems to perform better on IE, and a little worse on Chrome and others. However, the performance boost on IE is much greater than the slight degradation on Chrome (which again, isn't supposed to be inside the company.. but you know how those things work 😉

[Ricapar]

And as much as I'd love to make use of the REST API, that's too much dev work right now. Anyways, that's the reason why we got Splunk in the first place - to specifically avoid that type of dev work 🙂

But if it has to be done... might work as a longer-term solution further down the line.

[Ricapar]

@gkanapathy: I updated the post above with extra information. Users complaining about performance issues are all on IE9. I haven't any issues with Chrome or other WebKit browsers. As much as I wish I could tell all the users to switch to Chrome, Safari, Firefox... IE 9 is the corporate standard.

For now we're creating simpler views with less things going on and it is helping. That's the message that we're passing along now more-or-less.

[bmacias84]

I have also noticed users using IE have performance problems. Consider changing some of your charting to jscharts if possible. If you have a webproxy in your environment consider bypassing it by adding an exception for your Splunk Web servers.