If I log into the Splunk UI on a cluster peer (indexer), why don't I see all the data in the cluster in a search?

[cwl]

I have a 5.0.4 cluster environment with search factor (SF) and replication factor (RF) set as 2.

Since SF=2 both peers should have searchable copies of the hot bucket therefore I was expecting to get search results of these hot buckets from both peers' splunkweb directly.

When I did a test, I was able to get search result from the peer with the original hot bucket but I couldn't get search results from the peer that has the replicate copies of hot buckets.

Does anyone know if above behavior is by design ?

[hexx · Accepted solution]

By searching directly on the peers, you are breaking the contract of clustering.

Clustering only guarantees that you'll see your data correctly and with the appropriate level of redundancy when searching from the search-head. The peers don't have the whole picture - only the master and the search-head do.

Therefore, the behavior you are reporting is expected.