Deployment Architecture

How to deploy configurations to a Splunk 6.3.2 Search Head Cluster when a cluster member is down?

splunk_force_as
Path Finder

Hi,

I'm running a 4 node search head cluster where one search head is down due to hardware problems. When trying to deploy configurations to the SHC from the deployer, I get the following message:

Error while deploying apps to first member: ConfDeploymentException: Error while fetching apps baseline on target=https://host:port: Network-layer error: Connect Timeout. 

The first member is the host that is down. Is there a fix for this? Why wouldn't the deployer be able to push to the other members that are up? exception handling? bug? Any workarounds?

0 Karma
1 Solution

splunk_force_as
Path Finder

Unfortunately, if the first member is down, the splunk deployer will throw an error as I have seen. Per splunk support, this "fail fast" feature is in place to prevent configuration inconsistencies.

View solution in original post

0 Karma

phadnett_splunk
Splunk Employee
Splunk Employee

It is my understanding that the deployer fails-fast if the first member is down. If the second or later member is down, the deployer tries to push to remaining members, but then throws an error at the end. Essentially, there is no way to push a bundle if the first member is down.

The point behind this is that we do not want to perturb the system when a member is down, and we particularly don't want to create baseline configuration inconsistency.

splunk_force_as
Path Finder

Unfortunately, if the first member is down, the splunk deployer will throw an error as I have seen. Per splunk support, this "fail fast" feature is in place to prevent configuration inconsistencies.

View solution in original post

0 Karma

somesoni2
Revered Legend

For SHC bundle push you need URI of a SHC member which is up. If your first member is down, you should select second member which is up to push the bundle.

0 Karma

splunk_force_as
Path Finder

The target is a member that is not down. The deployer will still send to all cluster members regardless of the target. In this case, it's trying to send to the first member in the cluster that is down.

0 Karma

vin02
Path Finder

My all search head is up then also I am getting the same Error message. Could you please help me on this?

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!