topic How to forward data in multisite cluster in Splunk Search

How to forward data in multisite cluster

marco1987 — Fri, 17 Dec 2021 14:17:21 GMT

Hello splunkers,

i need to understand the best way to forward my data in multisite indexer cluster for Disaster Recovery management:

For example, we have:

On Site A

1 manager node (active)
3 peer nodes [IDX_1A, IDX_2A, IDX_3A ] (active)
1 search head (active)
2 Heavy Forwarder [HF_1A, HF_2A] (active)

On Site B

1 manager node (stand by)
3 peer nodes [IDX_1B, IDX_2B, IDX_3B ] (active)
1 search head (stand by)
2 Heavy Forwarder [HF_1B, HF_2B] (standy By)

On HF_1A and HF_2A the outputs.conf have to configure to send data to:

1) ALL site A and site B indexers (IDX_1A, IDX_2A, IDX_3A, IDX_1B, IDX_2B, IDX_3B)
we suppose that HF can comunicate with all

2) Only site A IDX? (IDX_1A, IDX_2A, IDX_3A)

3) Any other way?

Thanks in advance

Re: How to forward data in multisite cluster

richgalloway — Fri, 17 Dec 2021 15:51:17 GMT

Assuming the cost of inter-site network traffic is reasonable (nothing trans-oceanic, for instance) then have the forwarders send to all indexers. That will better distribute primary buckets and improve search performance through better map-reduce.

If site A forwarders send only to site A indexers then site B will contain only replicated buckets. The site B indexers will be less likely to participate in searches, so only 3 indexers instead of 6 will search.

Re: How to forward data in multisite cluster

isoutamo — Fri, 17 Dec 2021 16:18:50 GMT

here is described search affinity which @richgalloway already mentioned. https://docs.splunk.com/Documentation/Splunk/8.2.3/Indexer/Multisitesearchaffinity

As @richgalloway already said you need to know e.g. latency between sites to make decisions is another site only DR backup or will it participate also in normal usage.

r. Ismo