Deployment Architecture

How to resolve error "Error pulling configurations from the search head cluster captain"?

Path Finder

I am getting the error "Error pulling configurations from the search head cluster captain; consider performing a destructive configuration resync on this search head cluster member"

I tried to run the following command

# splunk resync shcluster-replicated-config

but i am getting the error "Cannot resync_destructive: this instance is the captain"

I then tried to perform the rolling restart among search head cluster, run the following command

# splunk rolling-restart shcluster-members

But still I am getting the error "Error pulling configurations from the search head cluster captain"
I also ran splunk resync shcluster-replicated-config after rolling-restart.
But still not fix. and I am getting above errors

Please suggest a fix

1 Solution

Path Finder

I am not getting the error now. I followed the below given action -

  1. stop the splunk on captain.
  2. deleted the files and directories under splunk/var/run/ after taking backup
  3. started the splunk on the search head.
  4. ran the resync command.

I have performed the above action 1 hour from now. I have not received any error as of now.

View solution in original post

Path Finder

Can someone please explain why this is an issue/ why deleting var/run is the best solution?

0 Karma

Path Finder
0 Karma

Communicator

Jut ran into this issue today after a big maintenance window this past week with lots of changes. This worked GREAT. Thank you everyone for the contributions. Awesome stuff!

0 Karma

Motivator

Running Splunk 7.1.1

the manual/destructive resync on the cluster member having the error corrected the issue for our cluster.

splunk resync shcluster-replicated-config

Path Finder

This worked for me! Thanks!

0 Karma

Engager

This worked for me. Tks

0 Karma

Path Finder

I am not getting the error now. I followed the below given action -

  1. stop the splunk on captain.
  2. deleted the files and directories under splunk/var/run/ after taking backup
  3. started the splunk on the search head.
  4. ran the resync command.

I have performed the above action 1 hour from now. I have not received any error as of now.

View solution in original post

Engager

Works on one of the members as well. I had a replication issue with one of the members, did the steps outlined here (on the member, not the captain) and it fixed it! 

Thanks! @mintughosh ! 🙂

0 Karma

Builder

Try transferring captain to another node and then perform resync.

https://docs.splunk.com/Documentation/Splunk/6.5.3/DistSearch/Transfercaptain#Change_the_captain

If this doesn't work, then restart the captain node and check.