Getting Data In

Why is one search head no longer communicating with indexers? Getting error "failed_because_BUNDLE_DATA_TRANSMIT_FAILURE"

vr2312
Contributor

We have around 10 Search Heads and 13 Indexers. Since this morning, we are seeing the below errors and our SH is not communicating with any of the Indexers associated.

All the other search heads are working as usual and have continuous communication to the Indexers.

05-23-2016 08:04:03.515 -0400 WARN  DistributedPeerManager - Unable to distribute to peer named XXX.YYY.ZZZ at uri https://XXX.YYY.ZZZ:8089 because replication was unsuccessful. replicationStatus Failed failure info: failed_because_BUNDLE_DATA_TRANSMIT_FAILURE
05-23-2016 08:04:03.515 -0400 WARN  DistributedPeerManager - Unable to distribute to peer named XXX.YYY.ZZZ at uri https://XXX.YYY.ZZZ:8089 because replication was unsuccessful. replicationStatus Failed failure info: failed_because_BUNDLE_DATA_TRANSMIT_FAILURE
05-23-2016 08:04:03.515 -0400 WARN  DistributedPeerManager - Unable to distribute to peer named XXX.YYY.ZZZ at uri https://XXX.YYY.ZZZ:8089 because replication was unsuccessful. replicationStatus Failed failure info: failed_because_BUNDLE_DATA_TRANSMIT_FAILURE
1 Solution

vr2312
Contributor

I just sorted the stuff on my own. It was not due to the server.conf file.

I noticed a lot of ".csv" were being transmitted to the indexers from that SH. And that has taken a toll on it.

I went ahead and blacklisted the files and the apps that are being replicated unnecessarily.

Now it works just like old times.

Thank you for your response @jkat54

View solution in original post

twinspop
Influencer

I had the same issue. I upgraded to 6.5.3 from 6.5.1 last week. This is the first time I've seen it. Possibly related? Anyway, a restart of the single search head that was reporting the problem fixed it. For now. Alert in place to catch more.

0 Karma

vr2312
Contributor

I just sorted the stuff on my own. It was not due to the server.conf file.

I noticed a lot of ".csv" were being transmitted to the indexers from that SH. And that has taken a toll on it.

I went ahead and blacklisted the files and the apps that are being replicated unnecessarily.

Now it works just like old times.

Thank you for your response @jkat54

jkat54
SplunkTrust
SplunkTrust

Please mark this as your answer... its ok to answer your own questions 😉

0 Karma

jkat54
SplunkTrust
SplunkTrust

Usually this is due to bad cluster password stored in pass4symmkey in server.conf.

0 Karma

twinspop
Influencer

Not the case here. A restart of the SH "fixed" it. No changes made.

0 Karma
Get Updates on the Splunk Community!

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...