Getting Data In

Is it possible to configure load balancing on universal forwarders with preferable servers in tcpout group?

rnr
Path Finder

I'd like to configure universal forwarders on boxes in multiple AZ to forward event to a preferable heavy forwarder located in the same AZ. The problem is word "preferable", universal forwarder doesn't have such settings (would be nice to have based on latency to forwarder for example).

Has anybody tried to build such setup using AWS tools, like Route 53 or ELB?
Router53 provides internal DNS, but I'm not sure if it can resolve names based on request source, that is AZ)

"Preferable" forwarder of course is not the end of the world, but would be nice to have.

--Roman

1 Solution

rnr
Path Finder

For those who interested in configuring local forwarders with prefferable destination and transparent failover, it's quite easy to do with haproxy.

For server located in zoneA:

{standard set of haproxy options}

frontend                            fe-splunkfwd
        mode                            tcp
        option                          tcplog
        log                             global
        bind                            127.0.0.1:7997
        default_backend                 be-splunkfwd


# For splunk forwarders
    backend                             be-splunkfwd
        mode                            tcp
        option                          tcplog
        timeout                         server 15s
        timeout                         connect 2s
        server                          fwd-server-name-zoneA  10.10.19.11:7997 maxconn 8192  check inter 1s
        server                          fwd-server-name-zoneB  10.19.20.11:7997 maxconn 8192  check inter 1s backup

Change backup option in the be-splunkfwd backend respectively for server locate in zoneB.

Of course it would work just fine with ELB, but on the other side local haproxy would give much better control of traffic. Nice to have it for a high logs throughput from host.

View solution in original post

0 Karma

rnr
Path Finder

For those who interested in configuring local forwarders with prefferable destination and transparent failover, it's quite easy to do with haproxy.

For server located in zoneA:

{standard set of haproxy options}

frontend                            fe-splunkfwd
        mode                            tcp
        option                          tcplog
        log                             global
        bind                            127.0.0.1:7997
        default_backend                 be-splunkfwd


# For splunk forwarders
    backend                             be-splunkfwd
        mode                            tcp
        option                          tcplog
        timeout                         server 15s
        timeout                         connect 2s
        server                          fwd-server-name-zoneA  10.10.19.11:7997 maxconn 8192  check inter 1s
        server                          fwd-server-name-zoneB  10.19.20.11:7997 maxconn 8192  check inter 1s backup

Change backup option in the be-splunkfwd backend respectively for server locate in zoneB.

Of course it would work just fine with ELB, but on the other side local haproxy would give much better control of traffic. Nice to have it for a high logs throughput from host.

0 Karma
Get Updates on the Splunk Community!

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...

Splunkbase | Splunk Dashboard Examples App for SimpleXML End of Life

The Splunk Dashboard Examples App for SimpleXML will reach end of support on Dec 19, 2024, after which no new ...

Understanding Generative AI Techniques and Their Application in Cybersecurity

Watch On-Demand Artificial intelligence is the talk of the town nowadays, with industries of all kinds ...