Some troubles when add host to SHC


Hi, guys.

I've Splunk Search Head cluster and I want add new member to this cluster. I use documentation from
After adding member KVstore doesn't synchronize with clustered KVStore. In log files I finded information about SSL certificate troubles.

2019-04-25T15:53:52.841Z I CONTROL [initandlisten] ** WARNING: No SSL certificate validation can be performed since no CA file has been provided
2019-04-25T15:53:52.841Z I CONTROL [initandlisten] ** Please specify an sslCAFile parameter.

I already generate new certs with splunk commands:

/opt/splunk/bin/splunk createssl server-cert -d /opt/splunk/etc/auth -n server -l 4096 -p

And add cacert info to server.conf

disabled = false
sslCAFile = /opt/splunk/etc/auth/cacert.pem

"splunk show kvstore-status" shows short read in log and not synchronized status

                     configVersion : -1
                       hostAndPort : host:8191
                     lastHeartbeat : Fri Apr 26 11:39:48 2019
                 lastHeartbeatRecv : ZERO_TIME
              lastHeartbeatRecvSec : 0
                  lastHeartbeatSec : 1556267988.693
                        optimeDate : ZERO_TIME
                     optimeDateSec : 0
                            pingMs : 0
                 replicationStatus : Down
                            uptime : 0

What can I do with this trouble?

0 Karma

Splunk Employee
Splunk Employee

Hi @GenRockeR ,

You can try to clean the local kvstore on the SH that you attempted to add to the SHC by following the steps below.

  1. Stop the search head that has the stale KV store member. (stop splunkd on
  2. Run the command splunk clean kvstore --local.
  3. Restart the search head. This triggers the initial synchronization from other KV store members.
  4. Run the command splunk show kvstore-status to verify synchronization.

The reference these steps can be found here:

0 Karma


Hi. I already try clean local KVstore, but still have troubles with replication. In mongod.log I found replication errors and short read connection status

0 Karma