Unable to log in to Deployment Server after upgrading Splunk

[boydtc]

Good day all,

I am having some issues after upgrading from Splunk Enterprise version 9.0.0 to 9.0.3. When log in to the deployment server and go to forwarder management, none of my data sources were listed and page was all blank white with nothing on there. I ensured the deployment server was enabled, checked firewalls which were ok, restarted Splunk and ensured Splunk was running which it was. No I am unable to log into the deployment server at all and gives the following errors and messages below:

Failed to contact license manager: reason='Unable to connect to license manager=https://hostname:8089 Error connecting: Connection refused'

[gcusello]

Hi @boydtc,

let me understand: you have in the same server the License Master and the Deployment Server?

Are there other roles on this Splunk server?

Ciao.

Giuseppe

[boydtc]

@gcusello 

I am sorry, it is the deployment server with the issue.

[gcusello]

Hi @boydtc,

all the Splunk servers (except Indexers) are usually configured as Forwarders (in the Enterprise License Group) and they forward their logs to indexers.

In your case, you connected the Deployment Server to the License master and it isn't correct.

Ciao.

Giuseppe

[boydtc]

Hmm ok, so we were able to resolve the part of the deployment server but now getting this error:

Error in Indexer Discovery communication. Verify that the pass4SymmKey set under [indexer_discovery:primary_indexers] in 'outputs.conf' matches the same setting under [indexer_discovery] in 'server.conf' on the cluster manager. [uri=cluster_manager_server:8089/services/indexer_discovery http_code=502 http_response="Error connecting: SSL not configured on client"]

I have already ensured the pass4symmkey was set but our web gives us the same message on our server when logging in the web:

Proxy Error

The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request GET

Reason: Error reading from remote server

Another list of error(s) was on the search head as well. It seems that however we updated from version 9.0.0 to 9.0.3 caused a misconfiguration maybe?

Failed to contact license manager: reason='Unable to connect to license managerlicense_manager_server:8089 Error connecting: Connection refused', first failure time=1672749015 (Tue Jan 3 07:30:15 2023 EST).

And:

  restricting search to internal indexes only (reason: [DISABLED_DUE_TO_GRACE_PERIOD,0])

It's just a connection issue between our cluster manager server and deployment server.

Thank you