I would like to build a Splunk multisite indexer cluster. However, I am concerned by the amount of network traffic generated if I enable replication between two sites. Therefore, I am considering disabling replication between sites. But in this case not all the data will be available locally within the site and there might be a situation where a search head from one site will be obtaining search results from a remote site. This will generate network traffic anyway and the latency might impact search performance. What do you think? What option is the best if I want to minimize traffic between the sites but at the same time maximize search performance?
Your Search Heads have a site-affinity. SH from site1 won't search on site2 unless it absolutely has to. So, in termins of search network traffic, you can relax.
In terms of your indexes and buckets: Having (for example) 2 buckets (1 + 1 copy) and another bucket on the second site won't generate that much traffic.
The question is, why do you want to set up a multi-site cluster? If you really need it, you may as well take advantage of the features it brings along.
Skalli