How do I enable mTLS for the replication port on a Search Head Cluster?

[OGS]

Please share your knowledge.

Splunk 9.4

reference
https://docs.splunk.com/Documentation/Splunk/9.4.2/Admin/Serverconf

I'm trying to set SHC replication to mTLS, but it's not working.
Alerts created in Splunk Web are being replicated.
I'm using a self-signed certificate.

search-head-1,search-head-2,search-head-3のsplunkd.log"port 9887 with SSL"is output.
08-06-2025 08:05:34.894 +0000 INFO TcpInputProc [148404 TcpListener] - Creating replication data Acceptor for IPv4 port 9887 with SSL

However, "useSSL=false" is output to all Search Heads.
08-08-2025 02:41:30.425 +0000 INFO SHCRepJob [21691 SHPPushExecutorWorker-0] - Running job=SHPRepJob peer="search-head-2", guid="A5CDBF4C-7F71-4705-9E20-10529800C25E" aid=scheduler__nobody_U3BsdW5rX1NBX0NJTQ__RMD5fe51f0ad1d9fe444_at_1754620680_13_A5CDBF4C-7F71-4705-9E20-10529800C25E, tgtPeer="search-head-1", tgtGuid="79BB42FF-7436-4966-B8C8-951EEF67C1AD", tgtRP=9887, useSSL=false

The correct response is returned with the openssl command.
The created self-signed certificate is also used on 8000 and 8089.

$ sudo openssl s_client \
-connect <host IP>:9887 \
-CAfile /opt/splunk/etc/auth/mycerts/<myRootCA>.pem \
-cert /opt/splunk/etc/auth/mycerts/<mycert>.pem \
-key /opt/splunk/etc/auth/mycerts/<mykey>.key

Verify return code: 0 (ok)

# /opt/splunk/etc/system/local/server.conf

[sslConfig]
enableSplunkdSSL = true
sslRootCAPath = /opt/splunk/etc/auth/mycerts/<myRootCA.pem>
serverCert = /opt/splunk/etc/auth/mycerts/<combined certificate.pem>
requireClientCert = true
sslVersions = tls1.2
sslCommonNameToCheck = <search-head-1>,<search-head-2>,<search-head-3>,・・・
sslPassword = <RootCR password>

[replication_port://9887]

[replication_port-ssl://9887]
disabled = false
serverCert = /opt/splunk/etc/auth/mycerts/<combined certificate.pem>
requireClientCert = true
sslVersions = tls1.2
sslCommonNameToCheck = <search-head-1>,<search-head-2>,<search-head-3>

I use Google Translate to translate Japanese into English.

[livehybrid · Accepted solution]

Hi @OGS 

You need to disable replication_port://9887 by either setting a disabled=true flag or ensuring it does not exist anywhere in your configuration - you can use btool to check:

 $SPLUNK_HOME/bin/splunk cmd btool server list --debug replication_port

If you have replication_port *and* replication_port-ssl enabled then this might conflict. 

Other things to note:

• serverCert must contain the server cert plus private key; sslPassword (if set) must be the private key’s passphrase (not the CA’s).
• sslRootCAPath must include the full trust chain (root + any intermediates).
• The names in sslCommonNameToCheck must match the CN/SANs in the peer certificates.

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[OGS]

Specifying it explicitly resolved the issue.

[replication_port://9887]
disabled = false

I confirmed from Captain's splunkd.log that communication with all tgtPeers is set to "useSSL=true."

Thank you.

[PickleRick]

The settings for tls should be set the same way as they are on management port. Your configuration looks more or less correct. What do you mean by "doesn't work"? Remember that you need to have a working CA for mTLS to work. Self-signed certs most probably won't work.

[OGS]

> The settings for TLS should be set the same way as they are on the management port.
Does this mean that it needs to match the port specified in mgmt_uri in the [shclustering] stanza?

> What do you mean by "doesn't work"?
> Remember that you need to have a working CA for mTLS to work.
> Self-signed certs most probably won't work.
The splunkd.log shows "useSSL=false," which goes against my intention.

This log result suggests that it's set to non-SSL.

I assumed that if communication was via mTLS, "useSSL=true" would be set.

If it doesn't work with a self-signed certificate, I'll try this setting another time.

Thank you for your advice.