Splunk Enterprise

Search Head Cluster: Failed HMAC signature match errors

a_kearney
Path Finder

Hi, I am running a Search Head Cluster with 7 search heads on Splunk 8.2.9.

2 of the search heads are generating the following error messages at ~5 second intervals for a period of time before stopping:

 

 

ERROR DigestProcessor [38271 TcpChannelThread] - Failed signature match
ERROR HTTPAuthManager [38271 TcpChannelThread] - Failed to verify HMAC signature, uri: /services/shcluster/member/consensus/pseudoid/raft_request_vote?output_mode=json

 

 

The search head cluster is otherwise running as expected as far as I can tell.

The search heads that are producing these errors are the only 2 that have been elected as captain in the last 30 days from examining the logs. There are no preferred captain or similar configurations set.

I have checked the [shclustering] pass4SymmKey values on each search head. They are all configured to the same value although use different Splunk Secrets to encrypt.

I am not sure when the errors first started appearing so can't link this to a specific upgrade on configuration change unfortunately.

The thread_id values seem to stay around for between 10-30 minutes. Sometimes 2 thread_ids will be active at once, sometimes none are active for a period. When looking at other logs for a particular thread_id around the same time period (at info logging level) I can't find see anything that adds any more cluses to what is causing the errors.

 

Labels (1)
Tags (1)
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...