Error [00000010] Instance name "indexer41.dev" Search head's authentication credentials rejected by peer. Try re-adding the peer. Last Connect Time:2018-07-10T16:07:50.000+00:00; Failed 1 out of 1 times.

[aksharp]

I'm just trying to connect 1 search head to 1 indexer.

Search head splunkd log

07-09-2018 17:21:46.755 +0000 WARN GetRemoteAuthToken - Unable to get authentication token from peeruri="https‍://indexer1.dev:8089/services/admin/auth-tokens".
07-09-2018 17:21:46.755 +0000 WARN DistributedPeer - Peer:https‍://indexer41.dev:8089 Authentication Failed
Indexer splunkd.log

07-09-2018 17:21:46.747 +0000 WARN AdminHandler:AuthenticationHandler - Denied session token for user: splunk-system-user
07-09-2018 17:21:46.755 +0000 WARN AdminHandler:AuthenticationHandler - Denied session token for user: splunk-system-user

Both have the same trusted.pem under the distsearchkeys directory. Both have the same admin passwords in $SPLUNK_HOME/etc/passwd.

Any help appreciated.

[renjith_nair]

Hi @aksharp ,

Usually it happens when your search peers (indexers) does not "trust" search head. It could be due to the rebuild of your server or migration to new server etc. To re-trust, as mentioned in the error, try re-adding the peers , (reference : https://docs.splunk.com/Documentation/Splunk/7.1.1/DistSearch/Configuredistributedsearch) which will overwrite the "stale" tokens/certs which were used to authenticate.