Splunk Search

search hangs post 4.3 upgrade

Path Finder

upgraded from 4.2.5 to 4.3 and now all searches timeout, and saved searches take longer to run. hw is 2x 4-core opteron 2374he, 32GB ram, single search head/web/indexer, index lives on fc connected SAN.

when trying to search, I include index, sourcetype, source to reduce search overhead, but load average hovers around 5 and all I get is: Timed out waiting for status to become available on job 12345678.

1,670,317,988 events indexed since 2/11/2011

couldn't find any changelog info on search process handling, I'm stumped, about to uninstall and reinstall 4.2.5

Tags (2)


Have you had a look inside the splunkd.log or searched it from the index=_internal within Splunk?

If you have alot of old indexed data there have been some people that have noticed Splunk performing some upgrade tasks on the buckets post upgrade which initially causes some slowdown. Also there have been some instances where the buckets are being checked for consistency constantly so its a good idea to have a look at the log to see if anything else is going on.
Perhaps install the SoS app to have a good look at anything that might be going on.

4.3 is faster than the previous versions so we just need to identify what is causing this slow-down at the moment.

0 Karma
Get Updates on the Splunk Community!

Observability | How to Think About Instrumentation Overhead (White Paper)

Novice observability practitioners are often overly obsessed with performance. They might approach ...

Cloud Platform | Get Resiliency in the Cloud Event (Register Now!)

IDC Report: Enterprises Gain Higher Efficiency and Resiliency With Migration to Cloud  Today many enterprises ...

The Great Resilience Quest: 10th Leaderboard Update

The tenth leaderboard update (11.23-12.05) for The Great Resilience Quest is out >> As our brave ...