After several years the replication factor on my 6.6.3 index cluster recently changed to 'not met'. it has been fine in the past, and I can see buckets replicating among the 4 members of the cluster...
See more...
After several years the replication factor on my 6.6.3 index cluster recently changed to 'not met'. it has been fine in the past, and I can see buckets replicating among the 4 members of the cluster. I'm able to see open connections on :9887 among the members, they show up in each others' splunkd.log as successful replications, and nothing has changed configuration-wise or even version-wise. I've got two members of the cluster with 200G less room than the other two (each), and I cannot find anything that can help me figure out what the problem is.
The monitoring console says 'Not Met', the show cluster-config says 'not met', and the --verbose output on the master looks like this for every entry, with the Replicated copies and Searchable copies trackers showing the same numbers all across the board:
network_wireless_aps
Number of non-site aware buckets=0
Number of buckets=28
Size=366897499
Searchable YES
Replicated copies tracker
28/28 28/28
Searchable copies tracker
28/28 28/28
network_wireless_controllers
Number of non-site aware buckets=0
Number of buckets=35
Size=19069528111
Searchable YES
Replicated copies tracker
35/35 35/35
Searchable copies tracker
35/35 35/35
(the same is true for the search-factor, but I figure if one gets better the other will too)