@Drainy is correct.
This problem is caused by a regression in Splunk 5.0 and 5.0.1 which impacts certain methods to list indexes, such as:
- The eventcount command
- REST API calls to the
/services/data/indexes endpoint
In 5.0 and 5.0.1, these methods fail to use cached assets and end up iterating over the files of your indexes themselves. This results in a much slower execution, which eventually affects whatever consumer calls the method.
In the Manager user interface, this affects some of the pages that need to present a list of indexes such as:
- The page showing the details of a saved search (Manager > Searches & Reports > $SAVED_SEARCH_NAME)
- The setup page for most data inputs (Manager > Data Inputs > Files & Directories > Add new)
- The page where you can list and edit indexes (Manager > Indexes)
- The page where you can edit roles (Manager > Access controls > Roles)
If the index-listing method takes more than 30s to complete, Splunkweb will time out on it and display a "500 - internal server error" message.
This issue will be fixed in maintenance release 5.0.2, which we are aiming to deliver early in 2013. We are also working on intermediary patch release 5.0.1.1 to specifically address this issue.
For reference, the bugs behind this problem are SPL-57518 and SPL-58650.
