Monitoring Splunk

After Deployment Server upgrade Manager Node is in sick state as Search Peer

lukasmecir
Path Finder

Hello,

I have distributed environment with IDX cluster and DS. DS is used for deploy config to IDX cluster Manager Node and from it to IDX cluster nodes then. It is working fine.

I upgraded DS from 8.1.6 to 8.1.10.1 (yes, because SVD-2022-0608...). Manager Node is on 8.1.6. After upgrade I noticed this log messages on MN:

10.88.28.93 - - [13/Jul/2022:15:56:33.540 +0200] "GET /services/server/info HTTP/1.1" 401 130 "-" "Splunk/8.1.10.1 (Linux 3.10.0-1160.62.1.el7.x86_64; arch=x86_64)" - 0ms

 10.88.28.93 is IP address of DS

I checked Search peers config on DS and there was MN in "sick" state. I edited its config by re-enter Remote username and Remote password and then MN changed status to Healthy and everything is working fine.

My question is: what happened during upgrade of DS? My idea is that new pair of private+public keys was generated on DS on first run after upgrade (and then I had to distribute new public key to MN by re-entering Remote username and password of course), but am I right? And if I am right, why this happened? I made many Splunk upgrades before and I experienced this never before...

Any info/hint/clue will be highly appreciated. Thank you.

Best regards

Lukas Mecir

Labels (1)
0 Karma
Get Updates on the Splunk Community!

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...