I constantly see the below error on my search head. What causes this and how do I go about fixing it. I have removed the IP address and replaced it with x.x.x.x:
Problem replicating config (bundle) to search peer 'x.x.x.x',Reading reply to upload: rv=-2, Receive from=https://x.x.x.x/8089 timed out; exceeded 60sec, as per=distsearch.conf/[replicationSettings]/sendRcvTimeout
Over time your Splunk install will get bigger, more apps added, more knowledge objects added leading to bigger knowledge bundles. The knowledge bundle is the data that the search head replicates and distributes to each search peer to enable its searches.
Have a look at the docs http://docs.splunk.com/Documentation/Splunk/6.3.1/DistSearch/Limittheknowledgebundlesize to learn about more on how to limit the size of the bundles. Also have a look at
distsearch.conf http://docs.splunk.com/Documentation/Splunk/6.3.1/admin/Distsearchconf and the option
sendRcvTimeout = <int, in seconds> * The maximum number of seconds to wait for the sending of a full replication to a peer.
to increase the number of seconds to wait for the replication.
Last but not least, check the network connection between your search heads and indexers if the bottleneck is there?
Is this just related to one search peer? If so, try to remove and add it again as search peer...
Hope this helps ...
I am getting the same error on my search head when I installed MLKT app on the search head . I didn't install the add-on on the indexer cluster . Is it mandatory for the addon to be installed on the indexer cluster.
Problem replicating config (bundle) to search peer '10.x.x.xx.xx:8089', error while transmitting bundle data.
I am getting that error on the search head after I enable the app on the search head without an addon on the indexer cluster. How can this be resolved and why is the bundle being pushed to the indexers while I install in the search head .
I wonder when Splunk will provide some real relief for this issue. We have many apps that require large lookups for accelerated data models and other indexer-level search functionality. There are many applications in the search head cluster that is connected to multiple indexer clusters that are developed and maintained by many different groups.
Bundle replication issues are the bane of my existence!!!!
Well I tried changing that setting to 600 seconds and I'm still getting this error.
I guess I'll dive into modifying the bundle to reduce the size. According to the logs it's only 30 MB. Both these servers are on the same subnet.
Seeing something similar in one of my deployments. Standalone search head talking to two clustered indexers.\
DistributedPeerManager - Unable to distribute to peer named at uri https://:8089 because replication was unsuccessful. replicationStatus Failed failure info: failed_because_BUNDLE_DATA_TRANSMIT_FAILURE
Same is happening on the second indexer too.