topic Getting error on Splunk restarts in the GUI after upgrading from 5.0 to 6.0 a few weeks ago. in Installation

Getting error on Splunk restarts in the GUI after upgrading from 5.0 to 6.0 a few weeks ago.

amgoldschmidt — Wed, 24 Jun 2020 19:30:47 GMT

I keep seeing this on Splunk restarts in the GUI

FAILED_LOAD_DEPLOYMENT_SERVER__TENANT_default

Anyone know what it is or why I am seeing it ?

I did upgrade from Splunk 5.0 to 6.0 a few weeks back, so might be related to that.

Re: FAILED_LOAD_DEPLOYMENT_SERVER__TENANT_default - keep seeing on restart

rgoody — Thu, 14 Nov 2013 18:07:08 GMT

Not sure but I am getting the same error after upgrade to version 6 so hopefully someone knows.

Re: FAILED_LOAD_DEPLOYMENT_SERVER__TENANT_default - keep seeing on restart

delink — Tue, 26 Nov 2013 14:49:18 GMT

Search for "ERROR DSManager" in your splunkd.log. You will likely find the specific reason that the deployment server is not able to start correctly. If you have trouble interpreting the logs, post them here so we can take a look.

Re: FAILED_LOAD_DEPLOYMENT_SERVER__TENANT_default - keep seeing on restart

ShaneNewman — Tue, 26 Nov 2013 16:30:40 GMT

11-25-2013 17:32:14.981 -0600 ERROR Application - Application=SA-ModularInput-PowerShell cannot be loaded, as path=C:\Program Files\Splunk\etc\deployment-apps\SA-ModularInput-PowerShell does not exist.
11-25-2013 17:32:14.981 -0600 ERROR DSManager - Failed to reload serverclass=MSSQL: Application=SA-ModularInput-PowerShell cannot be loaded, as path=C:\Program Files\Splunk\etc\deployment-apps\SA-ModularInput-PowerShell does not exist.
11-25-2013 17:32:14.981 -0600 WARN TenantService - Did not reload DSManager: Application=SA-ModularInput-PowerShell cannot be loaded, as path=C:\Program Files\Splunk\etc\deployment-apps\SA-ModularInput-PowerShell does not exist.
11-25-2013 17:32:14.981 -0600 ERROR loader - Deployment Server is not available, because errors prevented its initialization: 'Application=SA-ModularInput-PowerShell cannot be loaded, as path=C:\Program Files\Splunk\etc\deployment-apps\SA-ModularInput-PowerShell does not exist.'. You can try to reload Deployment Server after correcting the errors.
11-25-2013 17:32:14.981 -0600 INFO loader - win-service: Windows service is now in running state.

Re: FAILED_LOAD_DEPLOYMENT_SERVER__TENANT_default - keep seeing on restart

nwieseler — Wed, 27 Nov 2013 01:24:49 GMT

I had the same issue: updated from 5.0.4 to 6.0 and started seeing that error.

Going through the logs using Splunk on Splunk (S.O.S.) sorting by ERRORS I found this in the splunkd.log:

11/26/13
2:12:16.801 PM

11-26-2013 14:12:16.801 -0600 ERROR loader - Deployment Server is not available, because errors prevented its initialization: 'Gap in numbered regexes: expected attribute=attrib.3 not found (context: stanza='serverClass:class_01')'. You can try to reload Deployment Server after correcting the errors.

Using the config file viewer I used the chrome search function to find matching stuff (in my case "attrib") which turned up in the serverClass file. The issue was I had was I had skipped a number in the sequence (below, it goes 0 1 2 4 - should be 0 1 2 3). In the log you can see 'Gap in numbered regexes: expected attribute=attrib.3 not found (context: stanza='serverClass:class_01')'

snip

C:\Program Files\Splunk\etc\system\local\serverclass.conf attrib.0 = redacted00

C:\Program Files\Splunk\etc\system\local\serverclass.conf attrib.1 = redacted01

C:\Program Files\Splunk\etc\system\local\serverclass.conf attrib.2 = redacted02

C:\Program Files\Splunk\etc\system\local\serverclass.conf attrib.4 = redacted04

snip

Going back through my backups I noticed that it's been that way for a long time. Seems like something is checking this file for errors now (I can only speculate as I have no clue).

Hope this helps...

Nick

Re: FAILED_LOAD_DEPLOYMENT_SERVER__TENANT_default - keep seeing on restart

nwieseler — Wed, 27 Nov 2013 01:27:23 GMT

+1 for searching the logs... Splunk on Splunk makes this much easier (in my opinion anyway).

My issue was resolved viewing the splunkd.log file.

Re: FAILED_LOAD_DEPLOYMENT_SERVER__TENANT_default - keep seeing on restart

delink — Wed, 27 Nov 2013 18:37:13 GMT

Yes, SOS does make this a lot easier, but if that is not installed already, getting it installed might be more difficult just to solve this issue. Especially without a working deployment server! 🙂

Re: FAILED_LOAD_DEPLOYMENT_SERVER__TENANT_default - keep seeing on restart

amgoldschmidt — Mon, 28 Sep 2020 15:23:48 GMT

thanks, my error is "ERROR DSManager - Failed to reload serverclass=Unix: Application=unix_fwd cannot be loaded, as path=/opt/splunk/etc/deployment-apps/unix_fwd does not exist.
"

We are not even using the Deployment server.

Re: FAILED_LOAD_DEPLOYMENT_SERVER__TENANT_default - keep seeing on restart

delink — Mon, 02 Dec 2013 16:16:29 GMT

If you are not using the deployment server, you should clear out the etc/deployment-apps directory and make sure you remove any non-system-default serverclass.conf configuration file. This should effectively disable the deployment server in that instance of Splunk.

Be warned that you should also remove any deploymentclient.conf configuration file from forwarders or other Splunk instances that you are running so they no longer try to check in.

Re: FAILED_LOAD_DEPLOYMENT_SERVER__TENANT_default - keep seeing on restart

marksnelling — Thu, 05 Dec 2013 10:54:55 GMT

I'm also getting this problem since upgrading to Splunk 6.0. I have a generic windows serverClass and a serverClass for SQL server.
The SQL serverClass defines it's own repositoryLocation for the TA-SQLServer app, somehow my generic windows serverClass is also using this repositoryLocation to locate the SA-ModularInput-PowerShell app instead of the one defined in [global].

Here are the relevant sections of my serverclass.conf. I haven't found a solution to this yet.

[global]
repositoryLocation = $SPLUNK_HOME/etc/deployment-apps
targetRepositoryLocation = $SPLUNK_HOME/etc/apps
stateOnClient = enabled
restartSplunkd = true
whitelist.0 = *

[serverClass:Windows]
machineTypesFilter = windows-x64
whitelist.0 = *
[serverClass:Windows:app:Splunk_TA_windows]
[serverClass:Windows:app:SA-ModularInput-PowerShell]

[serverClass:MSSQL]
whitelist.0 = sql01
repositoryLocation = $SPLUNK_HOME/etc/apps/Splunk_for_SQLServer/appserver/addons
[serverClass:SQLServer:app:TA-SQLServer]

Re: FAILED_LOAD_DEPLOYMENT_SERVER__TENANT_default - keep seeing on restart

Adrian — Sat, 28 Dec 2013 16:28:53 GMT

I also upgrade and was presented with the same error. I believe it is an artifact left behind from previous installation. You have to comment out your entries in /opt/splunk/etc/system/local/serverclass.conf

The entry that was present in my serverclass.conf was the following:

[global]
# whitelist.0=* at the global level ensures that the machineTypesFilter attribute
# invoked later will apply.
whitelist.0=*

[serverClass:AppsByMachineType]
machineTypesFilter=linux-x86_64

[serverClass:AppsByMachineType:app:unix]
# Deploy this app only to unix boxes - 32/64 bit.
machineTypesFilter=linux-x86_64

After commenting it looked like this:

#[global]
# whitelist.0=* at the global level ensures that the machineTypesFilter attribute
# invoked later will apply.
#whitelist.0=*

#[serverClass:AppsByMachineType]
#machineTypesFilter=linux-x86_64

#[serverClass:AppsByMachineType:app:unix]
# Deploy this app only to unix boxes - 32/64 bit.
#machineTypesFilter=linux-x86_64

I then restarted /opt/splunk/bin/restart and the error message disappeared.

Re: FAILED_LOAD_DEPLOYMENT_SERVER__TENANT_default - keep seeing on restart

bosburn_splunk — Tue, 17 Jun 2014 13:47:54 GMT

I've found some issues around the Deployment Server and how it handles missing applications - which may be part of the problem here. I've opened bug SPL-85551 and will let you know what is found.