Deployment Architecture
Highlighted

Why does "splunk show shcluster-status" show "last_conf_replication: Pending" after upgrade from 6.3.2 to 6.5.0?

Explorer

Hi

Recently I upgraded my Splunk environment from 6.3.2 to 6.5.0. The environment has a search head cluster, an indexer cluster, and the manager server. After the upgrade, when I run the splunk show shcluster-status, I noticed there is a new status called lastconfreplication and it always shows Pending for the search head members, but the search head captain doesn't have this. The replication is running fine in the Indexer cluster, so I am not quite sure what this new option is about and how to get the replication finished. Can anyone shed some lights on this?

Thank you.

Highlighted

Re: Why does "splunk show shcluster-status" show "last_conf_replication: Pending" after upgrade from 6.3.2 to 6.5.0?

Splunk Employee
Splunk Employee

The docs describe the new lastconfreplication state field in the shcluster-status command:

"The lastconfreplication field indicates when the member last pulled a set of configurations from the captain"

http://docs.splunk.com/Documentation/Splunk/6.5.2/DistSearch/ViewSHCstatus#Show_cluster_status

So, it makes sense that the master will not have this status field. Not sure why all of the members would be stuck in a "Pending" state after an upgrade though. Maybe there is just a lot of config information to pull down? Are you seeing any other SHC related log messages that might indicate a problem?

0 Karma
Highlighted

Re: Why does "splunk show shcluster-status" show "last_conf_replication: Pending" after upgrade from 6.3.2 to 6.5.0?

Even for me also, other 2 Search Head Cluster member showing the same status.

lastconfreplication : Pending

Any quick suggestion to fixed it.

0 Karma
Highlighted

Re: Why does "splunk show shcluster-status" show "last_conf_replication: Pending" after upgrade from 6.3.2 to 6.5.0?

Explorer

We had the same state in our 8 member Searchhead cluster with Splunk 6.5.3. After consulting the Splunk Support it turned out, that we had to use the option "mgmturi" instead of "serverlist".
After switching the options the pending state disappeared and the cluster synched again.

This behaviour is a confirmed bug in 6.5.3. You can use either mgmturi or serverlist according to the documentation.

you also can use both options, but you will receive this warning message:
"“It is not advised to use serverslist and mgmturi at same time”"

0 Karma
Highlighted

Re: Why does "splunk show shcluster-status" show "last_conf_replication: Pending" after upgrade from 6.3.2 to 6.5.0?

Explorer

Hi markusspitzli2

Did Splunk Support advise in which release this bug will be fixed? Splunk 6.6 is recently released but when I went through the release note, I couldn't find the bug fix mentioned.

0 Karma
Highlighted

Re: Why does "splunk show shcluster-status" show "last_conf_replication: Pending" after upgrade from 6.3.2 to 6.5.0?

Explorer

Hi Daniel.

No. The support is still talking to the development team.

0 Karma