I can not connect to the search peer

[kawashita_t]

The following error message is output.

Error Message : Problem replicating config (bundle) to search peer 'IP:Port', can't establish http connection.

I thought that the bundle size is affecting and I created the following distsearch.conf file in / etc / sytem / local.
However, it did not solve it. Also, until the other day I was able to connect without problems.

[replicationSettings]
sendRcvTimeout = 120

[replicationWhitelist]
allConf = *.conf

[replicationBlacklist]
vr = apps/app1/...
risona = apps/app2/...

[distributedSearch]
servers = https://xx.xx.xx.xx:xxxx

The only change is that the search peer's license was exceeded.
Below is the contents of the splunkd.log

04-18-2017 10:27:38.787 +0900 INFO  NetUtils - Connect timeout - waited for 60 seconds. ip=xx.xx.xx.xx port=xxxx
04-18-2017 10:27:38.787 +0900 WARN  HTTPClient - Connect to=xx.xx.xx.xx:xxxx timed out; exceeded 60sec, as per=distsearch.conf/[replicationSettings]/connectionTimeout
04-18-2017 10:27:38.787 +0900 WARN  DistributedBundleReplicationManager - Bundle upload error: Connect to=https://xx.xx.xx.xx:xxxx timed out; exceeded 60sec, as per=distsearch.conf/[replicationSettings]/connectionTimeout
04-18-2017 10:27:38.787 +0900 ERROR DistributedBundleReplicationManager - Unable to upload bundle to peer named splunk01 with uri=https://xx.xx.xx.xx:xxxx.
04-18-2017 10:27:38.787 +0900 WARN  DistributedBundleReplicationManager - Asynchronous bundle replication to 1 peer(s) succeeded; however it took too long (longer than 10 seconds): elapsed_ms=63086, tar_elapsed_ms=2136, bundle_file_size=126300KB, replication_id=1492478795, replication_reason="async replication allowed"
04-18-2017 10:27:38.787 +0900 WARN  DispatchReaper - Spent 35559ms reaping bundle tarballs in $SPLUNK_HOME/var/run
04-18-2017 10:27:38.789 +0900 INFO  PipelineComponent - MetricsManager:probeandreport() took longer than seems reasonable (61310 milliseconds) in callbackRunnerThread. Might indicate hardware or splunk limitations.
04-18-2017 10:28:01.174 +0900 WARN  DistributedPeerManager - Unable to distribute to peer named splunk01 at uri https://xx.xx.xx.xx:xxxx because replication was unsuccessful. replicationStatus Failed failure info: failed_because_HTTP_CONNECTION_FAILURE

[suarezry · Accepted solution]

Delete the search peer from your distributed search config (in splunk web), then add the search peer back in. Does the replication succeed after this?

[kawashita_t]

Thank you for answer.
It was not a problem of Splunk, it was a network problem.

I want to investigate the network.