Getting Data In

What happens in a distributed environment with auto load balancing after the minFreeSpace is reached on an indexer?

mrg2k8
Explorer

Hello,

Having a distributed environment with N indexers and M servers sending data in a load-balanced way (autoLB=true) towards those indexers, what happens if the minimum free space is reached for one of the indexers?

The documentation says the indexer will not take any data until space is freed up and also "If one indexer is down, the forwarder immediately switches to another".

Will the load balancing algorithm on the servers just switch to the next indexer or will the data during that time frame ("The forwarder routes data to different indexers based on a specified time interval.") be lost?

1 Solution

bmacias84
Champion

Well that depends if you have Indexer acknowledgment enabled on your forwarders. If your Forwarder do not acknowledgement enabled any data send before the forwarder received an unable to connect will be dropped the floor. If the Indexer is has reached its minimum free space forwarders will be unable to connect and will attempt to connect to the next available Indexer. When a Forwarder has indexer acknowledgement enabled the forwarder will wait until Indexer has responded with an acknowledgement before clearing the event from the TCPOUT queue.

If you are worried about data loss enable indexer acknowledgement on the Forwarder, but you many have to also increase your TCPOUT queue beyond the default to prevent data from falling to the floor if your queue back up.

View solution in original post

bmacias84
Champion

Well that depends if you have Indexer acknowledgment enabled on your forwarders. If your Forwarder do not acknowledgement enabled any data send before the forwarder received an unable to connect will be dropped the floor. If the Indexer is has reached its minimum free space forwarders will be unable to connect and will attempt to connect to the next available Indexer. When a Forwarder has indexer acknowledgement enabled the forwarder will wait until Indexer has responded with an acknowledgement before clearing the event from the TCPOUT queue.

If you are worried about data loss enable indexer acknowledgement on the Forwarder, but you many have to also increase your TCPOUT queue beyond the default to prevent data from falling to the floor if your queue back up.

Get Updates on the Splunk Community!

Why You Can't Miss .conf25: Unleashing the Power of Agentic AI with Splunk & Cisco

The Defining Technology Movement of Our Lifetime The advent of agentic AI is arguably the defining technology ...

Deep Dive into Federated Analytics: Unlocking the Full Power of Your Security Data

In today’s complex digital landscape, security teams face increasing pressure to protect sprawling data across ...

Your summer travels continue with new course releases

Summer in the Northern hemisphere is in full swing, and is often a time to travel and explore. If your summer ...