Splunk Enterprise

Search Head Cluster: Failed HMAC signature match errors

a_kearney
Path Finder

Hi, I am running a Search Head Cluster with 7 search heads on Splunk 8.2.9.

2 of the search heads are generating the following error messages at ~5 second intervals for a period of time before stopping:

 

 

ERROR DigestProcessor [38271 TcpChannelThread] - Failed signature match
ERROR HTTPAuthManager [38271 TcpChannelThread] - Failed to verify HMAC signature, uri: /services/shcluster/member/consensus/pseudoid/raft_request_vote?output_mode=json

 

 

The search head cluster is otherwise running as expected as far as I can tell.

The search heads that are producing these errors are the only 2 that have been elected as captain in the last 30 days from examining the logs. There are no preferred captain or similar configurations set.

I have checked the [shclustering] pass4SymmKey values on each search head. They are all configured to the same value although use different Splunk Secrets to encrypt.

I am not sure when the errors first started appearing so can't link this to a specific upgrade on configuration change unfortunately.

The thread_id values seem to stay around for between 10-30 minutes. Sometimes 2 thread_ids will be active at once, sometimes none are active for a period. When looking at other logs for a particular thread_id around the same time period (at info logging level) I can't find see anything that adds any more cluses to what is causing the errors.

 

Labels (1)
Tags (1)
0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...