Splunk Web Interface Really Slow 10 Second Response Times

[barne_dn]

I'm trying to troubleshoot a slow splunk installation. It's painfully slow. I've deleted all indexes and starting from scratch. I've upgrade to the latest version as of this date, 4.3.4, build 136012.

My hardware is:

UltraSPARC-T1 @ 1.00 GHz
1 processor, 32 cores
16GB RAM.

I have one indexer and a secondary machine and I'm running distributed search.

When I click on ANY link, even if it's not a search link, it will take at least 10 seconds to respond, sometimes it takes up to 20 seconds or more. I've checked disk i/o and the processes to make sure nothing is getting bottlenecked, but these servers are only running splunk. I'm thinking this should be a lot faster, how can I diagnose this? I haven't been able to find any similar cases to go off from the splunk community.

[theouhuios]

Is this a Linux installation? Sometimes the issue can be because of other issues rather than just Splunk. If can be a load issue or memory issue on the servers. Check to see if they are fine when splunk is running. If you see something strange then probably check the logs. This is a general perspective from a Systems Admin point of view.

[barne_dn]

in solaris i use prstat. splunkd is between 4% - 6% consistently, everything else is pretty much at 0%. It's only using up 234M of mem, and I have 16GB. Which log files can I look at?

[theouhuios]

When you use a top/ ps -aux command does it show any information where load is something which you don't see in general? Also use free -m to see how much memory is free.

[barne_dn]

which logs? This is a sun solaris server. Splunk is the only thing running and the hardware should be more then enough.