Monitoring Splunk

After Deployment Server upgrade Manager Node is in sick state as Search Peer

lukasmecir
Path Finder

Hello,

I have distributed environment with IDX cluster and DS. DS is used for deploy config to IDX cluster Manager Node and from it to IDX cluster nodes then. It is working fine.

I upgraded DS from 8.1.6 to 8.1.10.1 (yes, because SVD-2022-0608...). Manager Node is on 8.1.6. After upgrade I noticed this log messages on MN:

10.88.28.93 - - [13/Jul/2022:15:56:33.540 +0200] "GET /services/server/info HTTP/1.1" 401 130 "-" "Splunk/8.1.10.1 (Linux 3.10.0-1160.62.1.el7.x86_64; arch=x86_64)" - 0ms

 10.88.28.93 is IP address of DS

I checked Search peers config on DS and there was MN in "sick" state. I edited its config by re-enter Remote username and Remote password and then MN changed status to Healthy and everything is working fine.

My question is: what happened during upgrade of DS? My idea is that new pair of private+public keys was generated on DS on first run after upgrade (and then I had to distribute new public key to MN by re-entering Remote username and password of course), but am I right? And if I am right, why this happened? I made many Splunk upgrades before and I experienced this never before...

Any info/hint/clue will be highly appreciated. Thank you.

Best regards

Lukas Mecir

Labels (1)
0 Karma
Get Updates on the Splunk Community!

Infographic provides the TL;DR for the 2024 Splunk Career Impact Report

We’ve been buzzing with excitement about the recent validation of Splunk Education! The 2024 Splunk Career ...

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...