Hello everyone,
After adding some data to Splunk and running apply cluster bundle & apply shcluster bundle commands, splunkd stopped working on one of our Search Heads (out of 2 in the cluster). We turned it on manually, but then it switched itself to a restart loop.
We have managed to solve the case with adjusting 'conf_deploy_fetch_mode' as suggested here: https://community.splunk.com/t5/Deployment-Architecture/Why-is-the-Search-Head-Cluster-getting-Caugh...
However, we are still looking for the exact reason of this issue. We have found the following internal errors on the affected SH:
04-15-2021 13:15:17.244 +0200 ERROR TcpInputProc - Error encountered for connection from src=myipaddress:port. Local side shutting down
04-15-2021 13:15:30.187 +0200 ERROR IndexAdminHandler - already reloading or shutting down, will not reload
Could someone please help me out interpreting them?
Thank you!