Getting Data In

Does external load balancer works with Universal/Heavy forwarder?

hrawat_splunk
Splunk Employee
Splunk Employee
0 Karma
1 Solution

hrawat_splunk
Splunk Employee
Splunk Employee

Outputs.conf 

External network load balancer will not work with UF/HF if DNS resolves only one IP-address for NLB. This limitation will be addressed in future releases.

Technically external network load balancer  should work with UF/HF as long as DNS returns 2 or more IP addresses.. So why not recommended?

There are some scenarios where results will not be as expected.

Let’s say we have 100 k FWDs. Assuming NLB dns lookup will return 2 A records(NLB-IP1, NLB-IP2).  Indexing tier with 25 indexers ( indexer1 - indexer25). Regardless of how NLB picks target indexer( volume or round robin or something else).
  1. UF1 connects to NLB-IP1 and NLB connects to indexer1.
  2. After autoLBFrequency(default 30 sec) UF1 will pick NLB-IP2. However It’s possible that NLB that is load balancing 100K FWD hosts, might connect to indexer1.
  3. Over the period of time let’s say 1 hour, it’s possible that UF1 (out of 120 connections) might ended up connecting to one indexer ( or few indexers) more than rest.

With fewer number of A records for NLB, forwarder might stick to only one indexer.

View solution in original post

0 Karma

hrawat_splunk
Splunk Employee
Splunk Employee

Outputs.conf 

External network load balancer will not work with UF/HF if DNS resolves only one IP-address for NLB. This limitation will be addressed in future releases.

Technically external network load balancer  should work with UF/HF as long as DNS returns 2 or more IP addresses.. So why not recommended?

There are some scenarios where results will not be as expected.

Let’s say we have 100 k FWDs. Assuming NLB dns lookup will return 2 A records(NLB-IP1, NLB-IP2).  Indexing tier with 25 indexers ( indexer1 - indexer25). Regardless of how NLB picks target indexer( volume or round robin or something else).
  1. UF1 connects to NLB-IP1 and NLB connects to indexer1.
  2. After autoLBFrequency(default 30 sec) UF1 will pick NLB-IP2. However It’s possible that NLB that is load balancing 100K FWD hosts, might connect to indexer1.
  3. Over the period of time let’s say 1 hour, it’s possible that UF1 (out of 120 connections) might ended up connecting to one indexer ( or few indexers) more than rest.

With fewer number of A records for NLB, forwarder might stick to only one indexer.

0 Karma
Get Updates on the Splunk Community!

Splunk APM & RUM | Upcoming Planned Maintenance

There will be planned maintenance of Splunk APM’s and Splunk RUM’s streaming infrastructure in the coming ...

Part 2: Diving Deeper With AIOps

Getting the Most Out of Event Correlation and Alert Storm Detection in Splunk IT Service Intelligence   Watch ...

User Groups | Upcoming Events!

If by chance you weren't already aware, the Splunk Community is host to numerous User Groups, organized ...