search hangs post 4.3 upgrade

[joshrabinowitz]

upgraded from 4.2.5 to 4.3 and now all searches timeout, and saved searches take longer to run. hw is 2x 4-core opteron 2374he, 32GB ram, single search head/web/indexer, index lives on fc connected SAN.

when trying to search, I include index, sourcetype, source to reduce search overhead, but load average hovers around 5 and all I get is: Timed out waiting for status to become available on job 12345678.

1,670,317,988 events indexed since 2/11/2011

couldn't find any changelog info on search process handling, I'm stumped, about to uninstall and reinstall 4.2.5

[Drainy]

Have you had a look inside the splunkd.log or searched it from the index=_internal within Splunk?

If you have alot of old indexed data there have been some people that have noticed Splunk performing some upgrade tasks on the buckets post upgrade which initially causes some slowdown. Also there have been some instances where the buckets are being checked for consistency constantly so its a good idea to have a look at the log to see if anything else is going on.
Perhaps install the SoS app to have a good look at anything that might be going on.

4.3 is faster than the previous versions so we just need to identify what is causing this slow-down at the moment.