Installation

Getting error on Splunk restarts in the GUI after upgrading from 5.0 to 6.0 a few weeks ago.

Explorer

I keep seeing this on Splunk restarts in the GUI

FAILED_LOAD_DEPLOYMENT_SERVER__TENANT_default

Anyone know what it is or why I am seeing it ?

I did upgrade from Splunk 5.0 to 6.0 a few weeks back, so might be related to that.

Labels (1)
1 Solution

Communicator

Search for "ERROR DSManager" in your splunkd.log. You will likely find the specific reason that the deployment server is not able to start correctly. If you have trouble interpreting the logs, post them here so we can take a look.

View solution in original post

Splunk Employee
Splunk Employee

I've found some issues around the Deployment Server and how it handles missing applications - which may be part of the problem here. I've opened bug SPL-85551 and will let you know what is found.

0 Karma

Path Finder

I also upgrade and was presented with the same error. I believe it is an artifact left behind from previous installation. You have to comment out your entries in /opt/splunk/etc/system/local/serverclass.conf

The entry that was present in my serverclass.conf was the following:

[global]
# whitelist.0=* at the global level ensures that the machineTypesFilter attribute
# invoked later will apply.
whitelist.0=*

[serverClass:AppsByMachineType]
machineTypesFilter=linux-x86_64

[serverClass:AppsByMachineType:app:unix]
# Deploy this app only to unix boxes - 32/64 bit.
machineTypesFilter=linux-x86_64

After commenting it looked like this:

#[global]
# whitelist.0=* at the global level ensures that the machineTypesFilter attribute
# invoked later will apply.
#whitelist.0=*

#[serverClass:AppsByMachineType]
#machineTypesFilter=linux-x86_64

#[serverClass:AppsByMachineType:app:unix]
# Deploy this app only to unix boxes - 32/64 bit.
#machineTypesFilter=linux-x86_64

I then restarted /opt/splunk/bin/restart and the error message disappeared.

0 Karma

Communicator

I'm also getting this problem since upgrading to Splunk 6.0. I have a generic windows serverClass and a serverClass for SQL server.
The SQL serverClass defines it's own repositoryLocation for the TA-SQLServer app, somehow my generic windows serverClass is also using this repositoryLocation to locate the SA-ModularInput-PowerShell app instead of the one defined in [global].

Here are the relevant sections of my serverclass.conf. I haven't found a solution to this yet.

[global]
repositoryLocation = $SPLUNK_HOME/etc/deployment-apps
targetRepositoryLocation = $SPLUNK_HOME/etc/apps
stateOnClient = enabled
restartSplunkd = true
whitelist.0 = *

[serverClass:Windows]
machineTypesFilter = windows-x64
whitelist.0 = *
[serverClass:Windows:app:Splunk_TA_windows]
[serverClass:Windows:app:SA-ModularInput-PowerShell]

[serverClass:MSSQL]
whitelist.0 = sql01
repositoryLocation = $SPLUNK_HOME/etc/apps/Splunk_for_SQLServer/appserver/addons
[serverClass:SQLServer:app:TA-SQLServer]
0 Karma

Path Finder

I had the same issue: updated from 5.0.4 to 6.0 and started seeing that error.

Going through the logs using Splunk on Splunk (S.O.S.) sorting by ERRORS I found this in the splunkd.log:

11/26/13
2:12:16.801 PM

11-26-2013 14:12:16.801 -0600 ERROR loader - Deployment Server is not available, because errors prevented its initialization: 'Gap in numbered regexes: expected attribute=attrib.3 not found (context: stanza='serverClass:class_01')'. You can try to reload Deployment Server after correcting the errors.

Using the config file viewer I used the chrome search function to find matching stuff (in my case "attrib") which turned up in the serverClass file. The issue was I had was I had skipped a number in the sequence (below, it goes 0 1 2 4 - should be 0 1 2 3). In the log you can see 'Gap in numbered regexes: expected attribute=attrib.3 not found (context: stanza='serverClass:class_01')'

snip

C:\Program Files\Splunk\etc\system\local\serverclass.conf attrib.0 = redacted00

C:\Program Files\Splunk\etc\system\local\serverclass.conf attrib.1 = redacted01

C:\Program Files\Splunk\etc\system\local\serverclass.conf attrib.2 = redacted02

C:\Program Files\Splunk\etc\system\local\serverclass.conf attrib.4 = redacted04

snip

Going back through my backups I noticed that it's been that way for a long time. Seems like something is checking this file for errors now (I can only speculate as I have no clue).

Hope this helps...

Nick

0 Karma

Motivator

11-25-2013 17:32:14.981 -0600 ERROR Application - Application=SA-ModularInput-PowerShell cannot be loaded, as path=C:\Program Files\Splunk\etc\deployment-apps\SA-ModularInput-PowerShell does not exist.
11-25-2013 17:32:14.981 -0600 ERROR DSManager - Failed to reload serverclass=MSSQL: Application=SA-ModularInput-PowerShell cannot be loaded, as path=C:\Program Files\Splunk\etc\deployment-apps\SA-ModularInput-PowerShell does not exist.
11-25-2013 17:32:14.981 -0600 WARN TenantService - Did not reload DSManager: Application=SA-ModularInput-PowerShell cannot be loaded, as path=C:\Program Files\Splunk\etc\deployment-apps\SA-ModularInput-PowerShell does not exist.
11-25-2013 17:32:14.981 -0600 ERROR loader - Deployment Server is not available, because errors prevented its initialization: 'Application=SA-ModularInput-PowerShell cannot be loaded, as path=C:\Program Files\Splunk\etc\deployment-apps\SA-ModularInput-PowerShell does not exist.'. You can try to reload Deployment Server after correcting the errors.
11-25-2013 17:32:14.981 -0600 INFO loader - win-service: Windows service is now in running state.

0 Karma

Communicator

Search for "ERROR DSManager" in your splunkd.log. You will likely find the specific reason that the deployment server is not able to start correctly. If you have trouble interpreting the logs, post them here so we can take a look.

View solution in original post

Communicator

If you are not using the deployment server, you should clear out the etc/deployment-apps directory and make sure you remove any non-system-default serverclass.conf configuration file. This should effectively disable the deployment server in that instance of Splunk.

Be warned that you should also remove any deploymentclient.conf configuration file from forwarders or other Splunk instances that you are running so they no longer try to check in.

0 Karma

Explorer

thanks, my error is "ERROR DSManager - Failed to reload serverclass=Unix: Application=unixfwd cannot be loaded, as path=/opt/splunk/etc/deployment-apps/unixfwd does not exist.
"

We are not even using the Deployment server.

0 Karma

Communicator

Yes, SOS does make this a lot easier, but if that is not installed already, getting it installed might be more difficult just to solve this issue. Especially without a working deployment server! 🙂

0 Karma

Path Finder

+1 for searching the logs... Splunk on Splunk makes this much easier (in my opinion anyway).

My issue was resolved viewing the splunkd.log file.

New Member

Not sure but I am getting the same error after upgrade to version 6 so hopefully someone knows.

0 Karma