Getting Data In

Load Balancing at Splunk

ibraheem
Explorer

With load balancing the Universal Forwarder sends data to all the indexers equally so that no indexer should get all the data and together the indexers holds all the data. It also provide automatic switchover capability incase of an indexer goes down.

Load balancing can be setup at UF in outputs.conf file in two ways:

 

  1. By time
  2. By Volume

 

For time based load balancing we used autoLBFrequency setting and for volume we use autoLBVolume.

Let's say I've three indexers on which I want to send data from UF. My outputs.conf file will look like below:

[tcpout: my_indexers] 
server=10.10.10.1:9997, 10.10.10.2:997, 10.10.10.3:9997

Now, to send data for 3 minutes to an indexer, then switch to another indexer and then to another, set the autoLBFrequency like this:

autoLBFrequency=180

Based on the above settings the UF will send data to indexer 10.10.10.1 for 3 minutes continuously then it will move towards the other indexers, and this loop will continue.

To send data based on the volume. Let's say to configure the UF to send 1MB data to an indexer then switch to another indexer in the list, the setting will look like below

autoLBVolume=1048576

In the cases of a very large file, such as a chatty syslog file, or loading a large amount of historical data, the forwarder may become "stuck" on one indexer, trying to reach EOF before being able to switch to another indexer. To mitigate this, you can use the forceTimebasedAutoLB setting on the forwarder. With this setting, the forwarder does not wait for a safe logical point and instead makes a hard switch to a different indexer every AutoLB cycle.

forceTimebasedAutoLB = true

To guard against loss of data when forwarding to an indexer you can enable indexer acknowledgment capability. With indexer acknowledgment, the forwarder will resend any data that the indexer does not acknowledge as "received". useACK setting is used for this purpose

useACK= true

The final output.conf will look like below

[tcpout]
useACK= true
autoLBFrequency=180
autoLBVolume=1048576

[tcpout: my_indexers] 
server=10.10.10.1:9997, 10.10.10.2:997, 10.10.10.3:9997
Labels (1)
0 Karma
Get Updates on the Splunk Community!

New This Month in Splunk Observability Cloud - Metrics Usage Analytics, Enhanced K8s ...

The latest enhancements across the Splunk Observability portfolio deliver greater flexibility, better data and ...

Alerting Best Practices: How to Create Good Detectors

At their best, detectors and the alerts they trigger notify teams when applications aren’t performing as ...

Discover Powerful New Features in Splunk Cloud Platform: Enhanced Analytics, ...

Hey Splunky people! We are excited to share the latest updates in Splunk Cloud Platform 9.3.2408. In this ...