Deployment Architecture

Why am I getting the following "send failure" message in my internal logs: "pushing PK to search peer" ?

jelizabeth
New Member

Here is the complete warning message:

Send failure while pushing PK to search peer = https://*.*.*.*:8089 , Read Timeout

I'm getting the above warning messages in the internal Splunk logs every minute from each of our 3 search heads.

The search peer in question is in our secondary site (let's say B) to the search heads (site A), but there are two other search peers in the same site (B) which we don't get any warning messages for.

I've done a ping and netcat from each of the search heads in site A to each of the three search peers in Site B and the results are the same for each one, connection established and similar ping times.

It's not a connection issue, so i'm wondering what else could be causing it?

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

It looks like you have some network issues between site A and site B (Maybe high latency). Same problem faced by other user previously and for them it was network issue. (reference : https://answers.splunk.com/answers/455635/why-is-my-search-head-cluster-not-working-after-up-1.html)

0 Karma

jelizabeth
New Member

As previously stated we don't believe it's a network issue as all tests between instances show no latency. we are looking for an alternative reason as to what could be causing the issue.

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

In that case you can directly distribute key files using process given here in Splunk Docs and after that check again whether splunk on Search Head in Site 1 is still complaining. If yes then I'll suggest to raise case with splunk support.

0 Karma

jelizabeth
New Member

thanks I'll give that a go

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...