My LM and MC are on the same server.  The MC shows all instances have Status Up including the CM server.  But from the MC Indexer Clustering Status, the Cluster Manager Not Reachable message is seen. We have Splunk Enteprise licensing that allows for Clustering. ... View more

Here are some more info: Verify the CM is still running -> yes CM is running Verify the CM is a search peer of the MC -> yes CM is search peer of the MC Confirm there are no pending changes waiting to be applied in the MC -> I do not see any pending changes Confirm the MC is still allowed to connect to the CM (new firewall rule, perhaps) -> I confirmed a curl from MC to CM is successful on tcp/8089 Check the MC's logs to see if reports any messages when trying to contact the CM.  From the CM I am seeing these messages:   8-10-2026 20:33:25.292 +0000 ERROR HttpClientRequest [7217 LMTrackerExecutorWorker-0] - HTTP client error=Connection refused while accessing server=https://<LM>:8089 for request=https://<LM>:8089/services/masterlm/usage. 08-10-2026 20:33:25.292 +0000 WARN HttpClientRequest [7217 LMTrackerExecutorWorker-0] - Returning error HTTP/1.1 502 Connection refused 08-10-2026 20:33:25.292 +0000 ERROR LMTracker [7217 LMTrackerExecutorWorker-0] - failed to send rows, reason='Unable to connect to license manager=https://<LM>:8089 Connection refused Restart the Splunk instance running the CM -> I restarted and even deleted the CM on the LM and re-added Restart the Splunk instance running the MC -> I restarted the MC ... View more

From what I can see in the latest Third-Party Package Updates in Splunk Enterprise - April 2026  2 Upgraded postgresql to version 17.7 to remedy CVE-2025-12817 and CVE-2025-12818 in Splunk Enterprise versions 10.2.2 and 10.0.5. Splunk Enterprise versions 9.4 and 9.3 are not affected Seems like Splunk Enterprise versions 10.2.2 and 10.0.5 mitigate CVE-2025-12817 and CVE-2025-12818, but I am not seeing any mention of remediating CVE-2026-2004, CVE-2026-2005, and CVE-2026-2006 in the latest Splunk Security Advisories.  I need to wait before updating and see which Splunk version brings postgres to version 17.8 ... View more

@richgalloway  I just upgraded to Splunk Enterprise 10.0.5 but I am still seeing the postgres vuln.  Do you know when postgres version 17.8 will be bundled with a new Splunk update? Path : /opt/splunk/bin/postgres Installed version : 17.7 Fixed version : 17.8 Third-Party Package Updates in Splunk Enterprise - April 2026 Package Remediation CVE Severity protobuf1 Upgraded protobuf to version 5.29.6 CVE-2026-0994 High postgresql2 Upgraded postgresql to version 17.7 Multiple Medium azure-core3 Upgraded azure-core to version 1.38.0 Multiple High OpenSSL4 Upgraded OpenSSL to version 1.0.2zo CVE-2026-22796 Low ... View more

See my latest message.  Also - I could have run this command to clear out the old/bad bundle... rm -rf $SPLUNK_HOME/var/run/splunk/deploy* ... View more

@PickleRick @JohnEGones @Zzyzx @LAME-Creations  I was able to follow this KB and it helped solve the issue for me Deployer fails to push a bundle to Search Head Cluster Peers in 9.4.1 (default apps) https://splunk.my.site.com/customer/s/article/Deployer-fails-to-push-a-bundle-to-Search-Head-Cluster-Peers-in-9-4-1 Changing the command arguments as per this Splunk KB Article prompted the Deployer to create a fresh staging directory, bypassing the stale/corrupt one that was causing the error. $SPLUNK_HOME/bin/splunk apply shcluster-bundle -target <URI>:<management_port> -push-default-apps true Also - I could have run this command to clear out the old/bad bundle... rm -rf $SPLUNK_HOME/var/run/splunk/deploy*   ... View more

I did not find a fix to this. ... View more

After reviewing again, the SHs were not restarting.  There was an issue with the SH replication port which I since fixed.  So now all the SHs show status: up.  But I am still having an issue with pushing a bundle.  I even removed all the current shcluster apps to a tmp directory and I created a dummy app and tried to push this with no luck. mkdir -p /opt/splunk/etc/shcluster/apps/_dummy_app/local echo "[ui]" > /opt/splunk/etc/shcluster/apps/_dummy_app/local/app.conf echo "is_visible = false" >> /opt/splunk/etc/shcluster/apps/_dummy_app/local/app.conf sudo chown -R splunk:splunk /opt/splunk Error while deploying apps to target=https://<SH-02>:8089 with members=3: Error while fetching apps baseline on target=https://<SH-02>:8089: Network-layer error: Read Timeout, Error while fetching apps baseline on target=https://<SH-03>:8089: Network-layer error: Read Timeout I also do not think this is a network or port issue since I can use this openssl command from the deployer to any of the SHs and I get what looks like a normal response.  I can also use the same command on SHs to the other SHs. openssl s_client -connect <IP address of SH>:8089    Not sure what to try next.  Thanks for your help! ... View more

Also - it seems that the SHs may be in a rolling restart loop?  splunk show shcluster-status --verbose shows the following: rolling_restart : restart status : Up -> status : Pending -> status : Up  ... View more

Yes - I was able to curl to the CM.  And I have the same decrypted pass4SymmKey.  I believe the issue was related to missing certs/SSL settings as you mention.  I was able to get this working.  Thanks. ... View more

Hello isoutamo, Thanks for your help!  I was able to log into one of the indexers and manually set frozenTimePeriodInSecs to a lower value.  This seemed to then allow me to Validate and Check, and then Push the new bundle from the Cluster Manager. So, it seems things are much more stable and the errors and warnings have disappeared.  But my indexers are still showing about 94% full for the /opt/splunk folder. ... View more

I narrowed the issue down to an add-on and then updated to the latest version.  This fixed the problem.  Thanks for you help @gcusello and @PickleRick  ... View more

Thanks @richgalloway  Yes - the outputs.conf on the SH did not have a reference to the SSL/TLS cert being used.  I added the path to the cert file and password.  it's now working 😀 ... View more

Hi PickleRick,   This is a lab/dev environment so we don't really have an RTO/RPO.  It would be great if you could list some options.  Thanks! ... View more

Thanks 13tsavage. I had to use this command to extract to /opt/splunk/etc: tar -xzvf /opt/$HOSTNAME.tgz -C / ... View more

Thanks 13tsavage This is just a standalone SH. So, I believe I should be OK with archiving $SPLUNK_HOME/etc Would this be a good command to be used to restore: tar xzvf /opt/$HOSTNAME.tgz -C /opt/splunk/etc/ ... View more