Recently I upgraded my Splunk environment from 6.3.2 to 6.5.0. The environment has a search head cluster, an indexer cluster, and the manager server. After the upgrade, when I run the
splunk show shcluster-status, I noticed there is a new status called lastconfreplication and it always shows Pending for the search head members, but the search head captain doesn't have this. The replication is running fine in the Indexer cluster, so I am not quite sure what this new option is about and how to get the replication finished. Can anyone shed some lights on this?
The docs describe the new lastconfreplication state field in the shcluster-status command:
"The lastconfreplication field indicates when the member last pulled a set of configurations from the captain"
So, it makes sense that the master will not have this status field. Not sure why all of the members would be stuck in a "Pending" state after an upgrade though. Maybe there is just a lot of config information to pull down? Are you seeing any other SHC related log messages that might indicate a problem?
Even for me also, other 2 Search Head Cluster member showing the same status.
lastconfreplication : Pending
Any quick suggestion to fixed it.
We had the same state in our 8 member Searchhead cluster with Splunk 6.5.3. After consulting the Splunk Support it turned out, that we had to use the option "mgmturi" instead of "serverlist".
After switching the options the pending state disappeared and the cluster synched again.
This behaviour is a confirmed bug in 6.5.3. You can use either mgmturi or serverlist according to the documentation.
you also can use both options, but you will receive this warning message:
"“It is not advised to use serverslist and mgmturi at same time”"
Did Splunk Support advise in which release this bug will be fixed? Splunk 6.6 is recently released but when I went through the release note, I couldn't find the bug fix mentioned.
No. The support is still talking to the development team.