Deployment Architecture

How to resolve error "Error pulling configurations from the search head cluster captain"?

mintughosh
Path Finder

I am getting the error "Error pulling configurations from the search head cluster captain; consider performing a destructive configuration resync on this search head cluster member"

I tried to run the following command

# splunk resync shcluster-replicated-config

but i am getting the error "Cannot resync_destructive: this instance is the captain"

I then tried to perform the rolling restart among search head cluster, run the following command

# splunk rolling-restart shcluster-members

But still I am getting the error "Error pulling configurations from the search head cluster captain"
I also ran splunk resync shcluster-replicated-config after rolling-restart.
But still not fix. and I am getting above errors

Please suggest a fix

1 Solution

mintughosh
Path Finder

I am not getting the error now. I followed the below given action -

  1. stop the splunk on captain.
  2. deleted the files and directories under splunk/var/run/ after taking backup
  3. started the splunk on the search head.
  4. ran the resync command.

I have performed the above action 1 hour from now. I have not received any error as of now.

View solution in original post

ridwanahmed
Path Finder

Can someone please explain why this is an issue/ why deleting var/run is the best solution?

0 Karma

ridwanahmed
Path Finder
0 Karma

joesrepsolc
Communicator

Jut ran into this issue today after a big maintenance window this past week with lots of changes. This worked GREAT. Thank you everyone for the contributions. Awesome stuff!

0 Karma

bandit
Motivator

Running Splunk 7.1.1

the manual/destructive resync on the cluster member having the error corrected the issue for our cluster.

splunk resync shcluster-replicated-config

ktwingstrom
Path Finder

This worked for me! Thanks!

0 Karma

linhmai_bne
Path Finder

This worked for me. Tks

0 Karma

mintughosh
Path Finder

I am not getting the error now. I followed the below given action -

  1. stop the splunk on captain.
  2. deleted the files and directories under splunk/var/run/ after taking backup
  3. started the splunk on the search head.
  4. ran the resync command.

I have performed the above action 1 hour from now. I have not received any error as of now.

mcazacu
Engager

Works on one of the members as well. I had a replication issue with one of the members, did the steps outlined here (on the member, not the captain) and it fixed it! 

Thanks! @mintughosh ! 🙂

0 Karma

dineshraj9
Builder

Try transferring captain to another node and then perform resync.

https://docs.splunk.com/Documentation/Splunk/6.5.3/DistSearch/Transfercaptain#Change_the_captain

If this doesn't work, then restart the captain node and check.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...