Why should search affinity be disabled in Multisite Search Head Cluster?

[gkas99]

We have multisite indexer cluster spanning across 2 DCs, one on west coast and another on east coast.
I am now working on the project to move from a single search head to multisite search head cluster setup.

I have trouble understanding what the benefit of turning off the search affinity in the SHC really is.

My understanding is that search affinity reduces traffic between sites because search heads only get results from indexers on their local site, meaning searches can run faster? (Ref: https://docs.splunk.com/Documentation/Splunk/8.2.2/Indexer/Multisitesearchaffinity)

However, this SHC documents, https://docs.splunk.com/Documentation/Splunk/8.2.2/DistSearch/DeploymultisiteSHC, recommends turning search affinity off so that:

• Search heads run searches across indexers spanning all sites
• If, instead, you set different search heads to different sites, the end user might notice lag time in getting some results, depending on which search head happens to run a particular search.

Well, wouldn't turning off search affinity make searches run slower if a search head gets results it needs from an indexer from another site?

It sounds to me like these 2 documentations contradict each other, unless I'm missing something.

[sbridge]

As usual, the answer is "it depends".   If you are not using a "streched" SH cluster, and all data is replicated and searchable in both sites, then you would want to turn on site affinity.  The two answers are using different assumptions, which they don't explain well, but both can be correct depending on your architecture.

The second answer is assuming you have a single streched SH cluster.

[isoutamo]

Hi

In your case when those DCs are quite long away each other, probably there is not a real advantage to disable this feature. But there are lot of installations when multisite clusters have implemented even a same DC or even same computer room for another reasons. Earlier one reason was that you could do "on line" update if you are using multisite. Currently this can do without it.

r. Ismo 

[gkas99]

Right, but what I still don't understand is the following point when having search affinity enabled in SHC:

• If, instead, you set different search heads to different sites, the end user might notice lag time in getting some results, depending on which search head happens to run a particular search.

In the case of SHC spanning 2 DCs that are far way from each other, with search affinity enabled, search heads get result from their local indexers anyway so there should not be slow responses? Unless, of course, the site becomes invalid and the search heads have to reach out to remote indexers, but that is an outage situation in which slow response is understandable.

In the case of SHC spanning 2 DCs in the close proximity, it doesn't really matter whether you have search affinity on or off because search heads should get very good responses from indexers on any site.

So in what scenario might users notice the lag?