When adding an indexer to a distributed environment, is there a configuration that makes indexers exchange events to auto load balance them?

[adamguzek]

Is there a configuration that makes indexers exchange events in order to auto load balance them? Let's say I add an indexer into distributed environment. I want to use it without reconfiguring syslog sources and forwarders.

Maybe it's a request - make indexers connect to each other, and move events between them to distribute in an optimal way...

Does indexer clustering with duplication of data give any advantage? Maybe then the search head is using first/second indexer to retrieve events... Not only "first copy"?

[dwaddle · Accepted solution]

In a distributed, non-clustered, environment the answer is a resounding 'no'. The various indexers have no knowledge of each other, there is no shared state across indexers. Both the search heads and the forwarders must be given knowledge of all the indexers. If you add an indexer and only configure it into your search head for distributed search, then it will get no data at all. You can't feed it data without making changes to your forwarders to send data to it.

When you enable clustering, this gives the indexer peers knowledge of each other, but only for the purpose of making redundant copies. An indexer can make an additional copy of data at a peer, but it cannot "migrate" its data to that peer. Come search time, an indexer bucket has but one primary copy, and it is only the primary copy that is searched. Any additional secondary copies do not participate in the search.

For the most part, the requirement that the forwarders know about all indexers does not change when you enable clustering. But, as of Splunk 6.3, the indexer discovery feature allows for forwarders to contact a cluster master and simply ask it "what indexers should I connect to?" Then when you add new indexers to the cluster, the forwarders learn of them automatically.