Splunk Search

How to forward data in multisite cluster

marco1987
Explorer

Hello splunkers,

i need to understand the best way to forward my data in multisite indexer cluster for Disaster Recovery management:

For example, we have:

On Site A

1 manager node (active)
3 peer nodes [IDX_1A, IDX_2A, IDX_3A ] (active)
1 search head (active)
2 Heavy Forwarder [HF_1A, HF_2A] (active)

On Site B

1 manager node (stand by)
3 peer nodes [IDX_1B, IDX_2B, IDX_3B ] (active)
1 search head (stand by)
2 Heavy Forwarder [HF_1B, HF_2B] (standy By)

On HF_1A and HF_2A the outputs.conf have to configure to send data to:

1) ALL site A and site B indexers (IDX_1A, IDX_2A, IDX_3A, IDX_1B, IDX_2B, IDX_3B)
we suppose that HF can comunicate with all

OR

2) Only site A IDX? (IDX_1A, IDX_2A, IDX_3A)

OR

3) Any other way?

Thanks in advance

Labels (1)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Assuming the cost of inter-site network traffic is reasonable (nothing trans-oceanic, for instance) then have the forwarders send to all indexers.  That will better distribute primary buckets and improve search performance through better map-reduce.

If site A forwarders send only to site A indexers then site B will contain only replicated buckets.  The site B indexers will be less likely to participate in searches, so only 3 indexers instead of 6 will search.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

Assuming the cost of inter-site network traffic is reasonable (nothing trans-oceanic, for instance) then have the forwarders send to all indexers.  That will better distribute primary buckets and improve search performance through better map-reduce.

If site A forwarders send only to site A indexers then site B will contain only replicated buckets.  The site B indexers will be less likely to participate in searches, so only 3 indexers instead of 6 will search.

---
If this reply helps you, Karma would be appreciated.

isoutamo
SplunkTrust
SplunkTrust

Hi

here is described search affinity which @richgalloway already mentioned. https://docs.splunk.com/Documentation/Splunk/8.2.3/Indexer/Multisitesearchaffinity

As @richgalloway already said you need to know e.g. latency between sites to make decisions is another site only DR backup or will it participate also in normal usage.

r. Ismo

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...