questions about splunk apply shcluster-bundle running a long time and eventually failed because of "Connection reset by" from Splunk deployer server.
Splunk 7.1.5. SH Cluster of 4 servers on AWS Linux
As a result I'm getting:
/opt/splunk/bin/splunk show shcluster-status
Splunk username: admin
Encountered some errors while trying to obtain shcluster status.
This node is not the captain of the search head cluster, and we could not determine the current captain. The cluster is either in the process of electing a new captain, or this member hasn't joined the pool
Found that splunk was not running on the SH Cluster captain and that 1 SH server was not responding to ssh
1) How to get the " splunk apply shcluster-bundle" command to provide more verbose feedback?
2) Like what SH server the bundle is being pushed to and what the % completion?
3) Any errors like SH server not reachable.
Luckily I was doing this in my test env, but how could the command be more smoothly ran?
You might want to use the Monitoring Console for analysing the details further. The MC comes with some nice dashbaords giving you an insight into your deployment, including deployment bundle sizes and how long it took to deploy.
A connection reset could have several root causes and without further logs, this is hard to isolate.