Splunk Enterprise

The splunkd daemon cannot be reached by splunkweb

splunkg
Explorer

Hello, since we upgraded to the version 10.4.3.0 we have encountered the problem that randomly after the login we hit the page with the following error message: 

Oops.

The splunkd daemon cannot be reached by splunkweb. Check that there are no blocked network ports or that splunkd is still running. Click here to return to Splunk homepage.

After a F5 Refresh the page loads again.. but that should not be the "solution".

After looking a bit trough the logs, the only error I found was this : 

2026-09-21 10:17:08,360 ERROR [6ab0e7e63b27b1af39010] decorators:357 - Splunkd daemon is not responding: ('Error connecting to /services/apps/local: The read operation timed out',)
Traceback (most recent call last):
File "D:\Splunk\Python-3.13\Lib\site-packages\splunk\rest\__init__.py", line 632, in simpleRequest
serverResponse, serverContent = h.request(uri, method, headers=headers, body=payload)
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

ERROR [6ab0e7e63b27b1af39010] __init__:659 - Socket error communicating with splunkd (error=The read operation timed out), path = /services/apps/local

Before the upgrade we did not had that problem. We did not change anything according to the firewall or ports.
Does anybody else encountered the same issue after upgrading ?

Labels (2)
0 Karma
1 Solution

livehybrid
SplunkTrust
SplunkTrust

Hi @splunkg 

I believe this might be a known issue, not a config problem on your end. Check out "Spike in 503 errors in the Splunk WebUI after upgrading to 10.0.3 (also affects 9.4.8/10.2.0)" - I think 10.4.3 has the same fault: 

Ultimately, recent releases added logic under [applicationsManagement] in server.conf that periodically pulls https://cdn.splunkbase.splunk.com/public/report/apps_dump.json to refresh the local apps cache used by /services/apps/local, even when allowInternetAccess = false. If your search head has no outbound internet access, the fetch hangs until it times out, and while it's hanging the /services/apps/local REST call that Splunk Web makes on every page load also stalls, throwing the "splunkd daemon cannot be reached" 503. The refresh/F5 "fix" works because the retry usually lands after the timeout has cleared.

Confirm it's the same fault by checking web_service.log around the timestamp for:

ERROR LocalAppsAdminHandler - Failed to fetch and parse apps dump; aborting splunkbase apps cache update

or

ERROR LocalAppsAdminHandler - Failed to parse apps dump uri from server.conf; aborting splunkbase apps cache update

There's a Splunk support KB specifically for this: "Error 503: Splunkd daemon cannot be reached by Splunk Web after upgrade to 9.4.8/10.x versions".

🌟 Did this answer help you? If so, please consider:

    • Adding karma to show it was useful
    • Marking it as the solution if it resolved your issue
    • Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing.

View solution in original post

livehybrid
SplunkTrust
SplunkTrust

Hi @splunkg 

I believe this might be a known issue, not a config problem on your end. Check out "Spike in 503 errors in the Splunk WebUI after upgrading to 10.0.3 (also affects 9.4.8/10.2.0)" - I think 10.4.3 has the same fault: 

Ultimately, recent releases added logic under [applicationsManagement] in server.conf that periodically pulls https://cdn.splunkbase.splunk.com/public/report/apps_dump.json to refresh the local apps cache used by /services/apps/local, even when allowInternetAccess = false. If your search head has no outbound internet access, the fetch hangs until it times out, and while it's hanging the /services/apps/local REST call that Splunk Web makes on every page load also stalls, throwing the "splunkd daemon cannot be reached" 503. The refresh/F5 "fix" works because the retry usually lands after the timeout has cleared.

Confirm it's the same fault by checking web_service.log around the timestamp for:

ERROR LocalAppsAdminHandler - Failed to fetch and parse apps dump; aborting splunkbase apps cache update

or

ERROR LocalAppsAdminHandler - Failed to parse apps dump uri from server.conf; aborting splunkbase apps cache update

There's a Splunk support KB specifically for this: "Error 503: Splunkd daemon cannot be reached by Splunk Web after upgrade to 9.4.8/10.x versions".

🌟 Did this answer help you? If so, please consider:

    • Adding karma to show it was useful
    • Marking it as the solution if it resolved your issue
    • Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing.

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Data Management Digest – September 2026

    Welcome to the September 2026 edition of Data Management Digest! September brought a fresh wave of ...

Federated Search for CloudWatch Unified Data Store Is Generally Available

As organizations modernize their cloud environments, AWS workloads generate more security, operational, and ...

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...