unable to distribute to peer - unable to get auth token - READ TIMEOUT

[bcyates]

We Just migrated two SH Vms to a new data center. Now we are having intermitten timeouts to where the search heads cannot distribute to peers and I am seeing this error message

WARN GetRemoteAuthToken - Unable to get auth token from peer: https://xx.xx.xxx:8089 due to: Read Timeout; exceeded 60000 milliseconds

[mstjohn_splunk]

Hi @bycyates. Did you figure out how to solve this problem? If so, would you mind describing what you did as an answer so other users could learn from your work? Thanks!

[mayurr98]

Hello
I suspect as you have migrated two SH Vms to a new data center there may be a connectivity issue
This usually appears when the search head cannot authenticate with search peers in the distributed environment. Pls re-authenticate either using CLI or from the GUI within the search head : settings-> distributed search -> Search peers and ensure, its up and enabled.

[deepashri_123]

Hey@bcyates,

This looks like authentication error. I think it is related to the distServerKeys being regenerated.
Also check the cluster pass4symmkey.

Let me know if this helps!!!