Deployment Architecture

How to deploy configurations to a Splunk 6.3.2 Search Head Cluster when a cluster member is down?

splunk_force_as
Path Finder

Hi,

I'm running a 4 node search head cluster where one search head is down due to hardware problems. When trying to deploy configurations to the SHC from the deployer, I get the following message:

Error while deploying apps to first member: ConfDeploymentException: Error while fetching apps baseline on target=https://host:port: Network-layer error: Connect Timeout. 

The first member is the host that is down. Is there a fix for this? Why wouldn't the deployer be able to push to the other members that are up? exception handling? bug? Any workarounds?

0 Karma
1 Solution

splunk_force_as
Path Finder

Unfortunately, if the first member is down, the splunk deployer will throw an error as I have seen. Per splunk support, this "fail fast" feature is in place to prevent configuration inconsistencies.

View solution in original post

0 Karma

phadnett_splunk
Splunk Employee
Splunk Employee

It is my understanding that the deployer fails-fast if the first member is down. If the second or later member is down, the deployer tries to push to remaining members, but then throws an error at the end. Essentially, there is no way to push a bundle if the first member is down.

The point behind this is that we do not want to perturb the system when a member is down, and we particularly don't want to create baseline configuration inconsistency.

splunk_force_as
Path Finder

Unfortunately, if the first member is down, the splunk deployer will throw an error as I have seen. Per splunk support, this "fail fast" feature is in place to prevent configuration inconsistencies.

0 Karma

somesoni2
Revered Legend

For SHC bundle push you need URI of a SHC member which is up. If your first member is down, you should select second member which is up to push the bundle.

0 Karma

splunk_force_as
Path Finder

The target is a member that is not down. The deployer will still send to all cluster members regardless of the target. In this case, it's trying to send to the first member in the cluster that is down.

0 Karma

vin02
Path Finder

My all search head is up then also I am getting the same Error message. Could you please help me on this?

0 Karma
Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...