Splunk Search

How to forward data in multisite cluster

marco1987
Explorer

Hello splunkers,

i need to understand the best way to forward my data in multisite indexer cluster for Disaster Recovery management:

For example, we have:

On Site A

1 manager node (active)
3 peer nodes [IDX_1A, IDX_2A, IDX_3A ] (active)
1 search head (active)
2 Heavy Forwarder [HF_1A, HF_2A] (active)

On Site B

1 manager node (stand by)
3 peer nodes [IDX_1B, IDX_2B, IDX_3B ] (active)
1 search head (stand by)
2 Heavy Forwarder [HF_1B, HF_2B] (standy By)

On HF_1A and HF_2A the outputs.conf have to configure to send data to:

1) ALL site A and site B indexers (IDX_1A, IDX_2A, IDX_3A, IDX_1B, IDX_2B, IDX_3B)
we suppose that HF can comunicate with all

OR

2) Only site A IDX? (IDX_1A, IDX_2A, IDX_3A)

OR

3) Any other way?

Thanks in advance

Labels (1)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Assuming the cost of inter-site network traffic is reasonable (nothing trans-oceanic, for instance) then have the forwarders send to all indexers.  That will better distribute primary buckets and improve search performance through better map-reduce.

If site A forwarders send only to site A indexers then site B will contain only replicated buckets.  The site B indexers will be less likely to participate in searches, so only 3 indexers instead of 6 will search.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

Assuming the cost of inter-site network traffic is reasonable (nothing trans-oceanic, for instance) then have the forwarders send to all indexers.  That will better distribute primary buckets and improve search performance through better map-reduce.

If site A forwarders send only to site A indexers then site B will contain only replicated buckets.  The site B indexers will be less likely to participate in searches, so only 3 indexers instead of 6 will search.

---
If this reply helps you, Karma would be appreciated.

isoutamo
SplunkTrust
SplunkTrust

Hi

here is described search affinity which @richgalloway already mentioned. https://docs.splunk.com/Documentation/Splunk/8.2.3/Indexer/Multisitesearchaffinity

As @richgalloway already said you need to know e.g. latency between sites to make decisions is another site only DR backup or will it participate also in normal usage.

r. Ismo

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...