Getting Data In

Does external load balancer works with Universal/Heavy forwarder?

hrawat_splunk
Splunk Employee
Splunk Employee
0 Karma
1 Solution

hrawat_splunk
Splunk Employee
Splunk Employee

Outputs.conf 

External network load balancer will not work with UF/HF if DNS resolves only one IP-address for NLB. This limitation will be addressed in future releases.

Technically external network load balancer  should work with UF/HF as long as DNS returns 2 or more IP addresses.. So why not recommended?

There are some scenarios where results will not be as expected.

Let’s say we have 100 k FWDs. Assuming NLB dns lookup will return 2 A records(NLB-IP1, NLB-IP2).  Indexing tier with 25 indexers ( indexer1 - indexer25). Regardless of how NLB picks target indexer( volume or round robin or something else).
  1. UF1 connects to NLB-IP1 and NLB connects to indexer1.
  2. After autoLBFrequency(default 30 sec) UF1 will pick NLB-IP2. However It’s possible that NLB that is load balancing 100K FWD hosts, might connect to indexer1.
  3. Over the period of time let’s say 1 hour, it’s possible that UF1 (out of 120 connections) might ended up connecting to one indexer ( or few indexers) more than rest.

With fewer number of A records for NLB, forwarder might stick to only one indexer.

View solution in original post

0 Karma

hrawat_splunk
Splunk Employee
Splunk Employee

Outputs.conf 

External network load balancer will not work with UF/HF if DNS resolves only one IP-address for NLB. This limitation will be addressed in future releases.

Technically external network load balancer  should work with UF/HF as long as DNS returns 2 or more IP addresses.. So why not recommended?

There are some scenarios where results will not be as expected.

Let’s say we have 100 k FWDs. Assuming NLB dns lookup will return 2 A records(NLB-IP1, NLB-IP2).  Indexing tier with 25 indexers ( indexer1 - indexer25). Regardless of how NLB picks target indexer( volume or round robin or something else).
  1. UF1 connects to NLB-IP1 and NLB connects to indexer1.
  2. After autoLBFrequency(default 30 sec) UF1 will pick NLB-IP2. However It’s possible that NLB that is load balancing 100K FWD hosts, might connect to indexer1.
  3. Over the period of time let’s say 1 hour, it’s possible that UF1 (out of 120 connections) might ended up connecting to one indexer ( or few indexers) more than rest.

With fewer number of A records for NLB, forwarder might stick to only one indexer.

0 Karma
Get Updates on the Splunk Community!

Build Scalable Security While Moving to Cloud - Guide From Clayton Homes

 Clayton Homes faced the increased challenge of strengthening their security posture as they went through ...

Mission Control | Explore the latest release of Splunk Mission Control (2.3)

We’re happy to announce the release of Mission Control 2.3 which includes several new and exciting features ...

Cloud Platform | Migrating your Splunk Cloud deployment to Python 3.7

Python 2.7, the last release of Python 2, reached End of Life back on January 1, 2020. As part of our larger ...