Deployment Architecture

Replicating kvstore to a new cluster


Hi guys.
Please help me with replicating kvstore to a new cluster member.
I already try resync kvstore but still have replication error.
In mongo log file I've some errors with short read statuses.

2019-06-05T15:49:12.460Z I ASIO     [NetworkInterfaceASIO-Replication-0] Connecting to splunk-sh02:8191
 2019-06-05T15:49:12.462Z I ASIO     [NetworkInterfaceASIO-Replication-0] Failed to connect to splunk-sh02:8191 - HostUnreachable: short read
 2019-06-05T15:49:12.462Z I ASIO     [NetworkInterfaceASIO-Replication-0] Dropping all pooled connections to splunk-sh02:8191 due to failed operation on a connection
 2019-06-05T15:50:00.638Z I REPL_HB  [replexec-214] Error in heartbeat (requestId: 1721194) to splunk-sh02:8191, response status: HostUnreachable: short read

bash-4.2$ /opt/splunk/bin/splunk show kvstore-status

 This member:
                       backupRestoreStatus : Ready
                                  disabled : 0
                                      guid : D9F9CB60-E02D-4B3A-8376-58E2A5DC8F02
                                      port : 8191
                                standalone : 0
                                    status : starting

 Enabled KV store members:
                                  guid : 6C58D8D2-2C37-492F-A14C-E32B7D55B040
                           hostAndPort : splunk-sh11:8191
                                  guid : 90730C74-4A10-423E-919B-6F88F01F3DF5
                           hostAndPort : splunk-sh21:8191
                                  guid : D9F9CB60-E02D-4B3A-8376-58E2A5DC8F02
                           hostAndPort : splunk-sh02:8191
0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!