topic Re: Should our two multisite clusters have distinct site numbers? in Monitoring Splunk

Should our two multisite clusters have distinct site numbers?

lycollicott — Tue, 11 Dec 2018 15:34:50 GMT

We have our original multisite cluster with site1 and site2. It will be decommissioned in 6 months when all of its indexes expire.

We built a new multisite cluster: should it be site 3 and site4? I think it should be, so that our search head cluster will be able to search both clusters.

Re: Should our two multisite clusters have distinct site numbers?

adonio — Tue, 11 Dec 2018 23:08:03 GMT

hello there,

i think that regardless its better to play it safe and give the new sites the numbers 3 and 4, unless you are considering moving data, which seems not to be the case here.
having said that, the SH relay on the Indexer Cluster Master to identify the relevant indexers to search from. meaning, considering you have a new Cluster Master, it is safe to have the new sites as site1 and site2 since they are connected to a new Cluster Master.
if you have a Monitoring Console (MC / DMC) you can review which indexer cluster (and sites) belong to each Cluster Master

Hope it helps

Re: Should our two multisite clusters have distinct site numbers?

lycollicott — Wed, 12 Dec 2018 13:21:29 GMT

We did a test at end of day yesterday with site1/2 and site 1/2, but the search head couldn't search the new site1/2 indexers. We're going to test more today, but I think I agree that site 3/4 is safer.

Re: Should our two multisite clusters have distinct site numbers?

adonio — Wed, 12 Dec 2018 14:28:52 GMT

are you using the same Cluster Master for the new Multisite Cluster?
if you consider the question as answered, kindly mark it us such for others to know

Re: Should our two multisite clusters have distinct site numbers?

lycollicott — Wed, 12 Dec 2018 15:17:19 GMT

Using site3/4 for the new cluster does work either.

Cluster A

Cluster Master -> site1
Cluster Peer -> site1
Cluster Peer -> site2

Cluster B

Cluster Master -> site3
Cluster Peer -> site3
Cluster Peer -> site3
Cluster Peer -> site4
Cluster Peer -> site4

We'll keep testing today.

Re: Should our two multisite clusters have distinct site numbers?

adonio — Wed, 12 Dec 2018 15:37:02 GMT

do you have search affinity enabled?
https://docs.splunk.com/Documentation/Splunk/latest/Indexer/Multisitesearchaffinity
if you do, disable it.
also, are you using a separate Indexer Cluster Master for new Multisite Cluster?
did you connect the Search Head/s to the new Cluster Master?

Re: Should our two multisite clusters have distinct site numbers?

lycollicott — Wed, 12 Dec 2018 20:37:40 GMT

After a little more testing today, we have search working against both indexer clusters with site1 - site4.

Re: Should our two multisite clusters have distinct site numbers?

lycollicott — Thu, 13 Dec 2018 13:54:50 GMT

Yes, we did both of those things. We started by testing on a single search head and that was successful yesterday, so we're actually pointing the rest of the SHC there right now. I'm really happy with how it's working.