Deployment Architecture

Unable to log in to Deployment Server after upgrading Splunk

boydtc
Loves-to-Learn Everything

Good day all,

I am having some issues after upgrading from Splunk Enterprise version 9.0.0 to 9.0.3. When log in to the deployment server and go to forwarder management, none of my data sources were listed and page was all blank white with nothing on there. I ensured the deployment server was enabled, checked firewalls which were ok, restarted Splunk and ensured Splunk was running which it was. No I am unable to log into the deployment server at all and gives the following errors and messages below:

boydtc_0-1672846034176.png

Failed to contact license manager: reason='Unable to connect to license manager=https://hostname:8089 Error connecting: Connection refused'

0 Karma

woodcock
Esteemed Legend

This post has degenerated into "I have many errors, please fix them all".  You need to decide on which problem you need to tackle one at a time and STICK to that one problem in any post here.  If it turns into or uncovers another problem, then post a new question.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @boydtc,

let me understand: you have in the same server the License Master and the Deployment Server?

Are there other roles on this Splunk server?

Ciao.

Giuseppe

0 Karma

boydtc
Loves-to-Learn Everything

@gcusello 

I am sorry, it is the deployment server with the issue.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @boydtc,

all the Splunk servers (except Indexers) are usually configured as Forwarders (in the Enterprise License Group) and they forward their logs to indexers.

In your case, you connected the Deployment Server to the License master and it isn't correct.

Ciao.

Giuseppe

0 Karma

boydtc
Loves-to-Learn Everything

Hmm ok, so we were able to resolve the part of the deployment server but now getting this error:

Error in Indexer Discovery communication. Verify that the pass4SymmKey set under [indexer_discovery:primary_indexers] in 'outputs.conf' matches the same setting under [indexer_discovery] in 'server.conf' on the cluster manager. [uri=cluster_manager_server:8089/services/indexer_discovery http_code=502 http_response="Error connecting: SSL not configured on client"]

 

I have already ensured the pass4symmkey was set but our web gives us the same message on our server when logging in the web:

Proxy Error

The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request GET

Reason: Error reading from remote server

 

Another list of error(s) was on the search head as well. It seems that however we updated from version 9.0.0 to 9.0.3 caused a misconfiguration maybe?

Failed to contact license manager: reason='Unable to connect to license managerlicense_manager_server:8089 Error connecting: Connection refused', first failure time=1672749015 (Tue Jan 3 07:30:15 2023 EST).

And:

  restricting search to internal indexes only (reason: [DISABLED_DUE_TO_GRACE_PERIOD,0])

It's just a connection issue between our cluster manager server and deployment server.

Thank you

0 Karma

EricH92
Observer

Did you ever figure out the fix for the Proxy error?

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...