Splunk Enterprise Security

Need help with reconfiguring LM and DMC on rebuilt linux server

sdkp03
Communicator

We did rebuild existing server that hosted LM and DMC. I did install latest splunk on the rebuilt server. Copied configuration settings from old build. Now when I try to turn  on splunk service, the service does start up fine but when I try accessing it from UI, I see below mentioned errors logged in splunkd.log:

Error in Indexer Discovery communication. Verify
that the pass4SymmKey set under [indexer_discovery:dr_indexer_cluster_group] in 'outputs.conf' matches the same setting
under [indexer_discovery] in 'server.conf' on the Cluster Master. [uri=https://splunk-idx-cm.trgtm.ostravam.corp.tels
tra.com:8089/services/indexer_discovery http_code=502 http_response="Error connecting: SSL not configured on client"]

I did decrypt and verified that the pass4symmkey is same on CM and the new server. 

> SSLCommon - Can't read key file /opt/splunk/etc/auth/server.pem errno=101077092 err
or:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt

SSL context could not be created - error in cert or password is wrong
 HTTPServer - SSL will not be enabled

Not sure what this error is all about.

 

Labels (3)
Tags (1)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Make sure the new server has the correct SSL certificate(s) installed.

---
If this reply helps you, Karma would be appreciated.
0 Karma

sdkp03
Communicator

Thanks for the reply. I have been thinking on same lines and trying to find steps to install new certificates. However I did not find any steps to establish connection between DMC/LM and peers. Error message as extracted from our log file:

ERROR SSLCommon - Can't read key file /opt/splunk/etc/auth/ost_certs/model_lms_CA.pem errno=101077092 error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt

 

I have verified that the license file has no special characters. Have also verified that the existing cert can be accessed using below command:

openssl x509 -in /opt/splunk/etc/auth/ost_certs/model_lms_CA.pem -text -noout

 

Do you think following same steps as below would help me with the issue?

https://community.splunk.com/t5/Security/How-do-I-set-up-SSL-forwarding-with-new-self-signed-certifi...

0 Karma

richgalloway
SplunkTrust
SplunkTrust

I don't have enough experience with SSL to say for sure, but I think it's worth trying.

---
If this reply helps you, Karma would be appreciated.
0 Karma

isoutamo
SplunkTrust
SplunkTrust

Have you check that your cert file contains the whole certificate chain?

Can you send your server.conf and outputs.conf?

r. Ismo

0 Karma

sdkp03
Communicator

Eventually it has been found that the issue has been identified with the sslpassword. Old certificates were configured with sslpassword that started with a space. Example: " test123". Now when I try to store the same password in outputs.conf and restart the server, splunk is ignoring the leading space character. I have tried to add password in double quotes, single quotes, escaping with a forward slash - nothing seems to be helping with getting the right password encrypted. This is where am stuck now at. 

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

Can you try to change password of this certificate file? E.g. https://smallbusiness.chron.com/change-pass-phrase-ssl-certificates-29137.html

r. Ismo

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...