Getting Data In

Does external load balancer works with Universal/Heavy forwarder?

hrawat_splunk
Splunk Employee
Splunk Employee
0 Karma
1 Solution

hrawat_splunk
Splunk Employee
Splunk Employee

Outputs.conf 

External network load balancer will not work with UF/HF if DNS resolves only one IP-address for NLB. This limitation will be addressed in future releases.

Technically external network load balancer  should work with UF/HF as long as DNS returns 2 or more IP addresses.. So why not recommended?

There are some scenarios where results will not be as expected.

Let’s say we have 100 k FWDs. Assuming NLB dns lookup will return 2 A records(NLB-IP1, NLB-IP2).  Indexing tier with 25 indexers ( indexer1 - indexer25). Regardless of how NLB picks target indexer( volume or round robin or something else).
  1. UF1 connects to NLB-IP1 and NLB connects to indexer1.
  2. After autoLBFrequency(default 30 sec) UF1 will pick NLB-IP2. However It’s possible that NLB that is load balancing 100K FWD hosts, might connect to indexer1.
  3. Over the period of time let’s say 1 hour, it’s possible that UF1 (out of 120 connections) might ended up connecting to one indexer ( or few indexers) more than rest.

With fewer number of A records for NLB, forwarder might stick to only one indexer.

View solution in original post

0 Karma

hrawat_splunk
Splunk Employee
Splunk Employee

Outputs.conf 

External network load balancer will not work with UF/HF if DNS resolves only one IP-address for NLB. This limitation will be addressed in future releases.

Technically external network load balancer  should work with UF/HF as long as DNS returns 2 or more IP addresses.. So why not recommended?

There are some scenarios where results will not be as expected.

Let’s say we have 100 k FWDs. Assuming NLB dns lookup will return 2 A records(NLB-IP1, NLB-IP2).  Indexing tier with 25 indexers ( indexer1 - indexer25). Regardless of how NLB picks target indexer( volume or round robin or something else).
  1. UF1 connects to NLB-IP1 and NLB connects to indexer1.
  2. After autoLBFrequency(default 30 sec) UF1 will pick NLB-IP2. However It’s possible that NLB that is load balancing 100K FWD hosts, might connect to indexer1.
  3. Over the period of time let’s say 1 hour, it’s possible that UF1 (out of 120 connections) might ended up connecting to one indexer ( or few indexers) more than rest.

With fewer number of A records for NLB, forwarder might stick to only one indexer.

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Customer Survey!

If you use Splunk Observability Cloud, we invite you to share your valuable insights with us through a brief ...

Happy CX Day, Splunk Community!

Happy CX Day, Splunk Community! CX stands for Customer Experience, and today, October 3rd, is CX Day — a ...

.conf23 | Get Your Cybersecurity Defense Analyst Certification in Vegas

We’re excited to announce a new Splunk certification exam being released at .conf23! If you’re going to Las ...