Deployment Architecture

High Availability - Light Forwarders and combining cloning and autoLB

Glenn
Builder

We are looking to improve our current Splunk architecture (which is the same as the example in "Data cloning for high availability from http://www.splunk.com/wiki/Community:MultipleIndexServerDeploymentOptions, with one indexer each in two datacentres), to something capable of handling a higher volume, after we recently purchased larger Splunk licenses for the business.

This question is about the bottom end of the architecture, how the LightForwarders pass their data up the system to what we have already (provisionally) decided will be a “cluster” of indexers in each datacentre, being fed by an autoLB source. Searching will be done via a distributed search head in each DC, searching all of the "cluster" nodes in that same DC.

To do this, we need data to be cloned across both datacentre indexer clusters, but the data to be shared between cluster nodes via autoLB.

LightForwarders are required on the source servers, because of their small footprint. I want to know if this is possible for them.

outputs.conf:

[tcpout]
defaultGroup=dc1,dc2

[tcpout:dc1]
autoLB=true
autoLBFrequency=30
server=splunkindexers.dc1.company.com:42099

[tcpout:dc2]
autoLB=true
autoLBFrequency=30
server=splunkindexers.dc2.company.com:42099

Where splunkindexers.dc1.companyname.com is a DNS list that contains:
- indexer1.dc1.company.com
- indexer2.dc1.company.com

…and splunkindexers.dc2.companyname.com is a DNS list that contains:
- indexer3.dc2.company.com
- indexer4.dc2.company.com

This would combine the LightForwarder capability of cloning with the LightForwarder capability of autoLB to DNS lists, giving us what we want (cloning to two indexer clusters(one in each DC), but spreading the data between cluster nodes). Can LWFs handle the combination of its two capabilities at once?

Since there is no requirement to process any of the data (which is not possible with a LWF), and it only uses forwarding strategies that the LightForwarder is capable of (albeit a combination of them), I am hoping that it is… if it can do them separately why couldn’t it do both?

Cheers,

Glenn

1 Solution

jkerai
Splunk Employee
Splunk Employee

Yes, this configuration should work. You should be able to clone the data to 2 clusters of indexers for HA.

View solution in original post

jkerai
Splunk Employee
Splunk Employee

Yes, this configuration should work. You should be able to clone the data to 2 clusters of indexers for HA.

Glenn
Builder

Dang, now I find that it has already mostly been answered in http://answers.splunk.com/questions/421/is-it-possible-to-configure-cloning-and-autolb-simultaneousl... - and it looks like it is possible. Anyway, if anyone has a confirmation, or any useful comments for my new architecture proposale, it would be much appreciated.

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...