Load Balance Priorities

[chrislymanWMT1]

Okay, stick with me on this one. Take a scenario where I have two sites, one a central datacenter and the other a remote office location connected via a somewhat congested WAN. I also have two indexers at each location with distributed search pointing to all four. What I would like to configure is the forwarders at the remote location always try to send their data to the closest indexer (i.e. not transverse the WAN), but if both of the local indexers are down then I want the forwarder to attempt to send the data to the indexers at the datacenter. In a nutshell I want the forwarders to prioritize one group of indexers over another group. This is important for us in terms of indexing audit data that must be indexed somewhere. Is this possible?

Any input would be appreciated.

[adamw]

This is not currently possible. I'd recommend you have the local forwarders send to their local indexers all the time.

If you have available resources, you could do the following:

DC1Forwarders -> datacenter1indexer1 -> datacenter2indexer1
DC2Forwarders -> datacenter2indexer2 -> datacenter1indexer2

This way the forwarders always talk to their local indexers, then the indexers do the forwarding over the WAN link to each other, allowing you to better configure queueing on the indexers instead of on the forwarders. This also allows you to set up two HA searchheads, one in datacenter1 peered to datacenter1's indexers, and one in datacenter2 peered to datacenter2's indexers.

This is the recommended way of doing HA across datacenters, until Splunk releases multi-datacenter replication.

[adamw]

If this doesn't make sense, I can draw a better ascii diagram.