Getting Data In
Highlighted

Recommended load balancer for indexer

Communicator

Hi splunkers,

I just want to ask for any recommended or even tested loadbalancer upon forwarding logs to 3 indexers. My current setup is I have 3 indexers and I need my fortigate,cisco switch and some linux box logs to be forwarder on those 3 indexers in a round robin balancing.

Thanks,

0 Karma
Highlighted

Re: Recommended load balancer for indexer

Builder

You will find that it is well-documented here;

http://docs.splunk.com/Documentation/Splunk/6.2.0/Forwarding/Setuploadbalancingd

Use the built in functions in the splunk forwarder to loadbalance your data or with dns .

Highlighted

Re: Recommended load balancer for indexer

Communicator

Hi Imyrefelt,

Thanks for your help. So the easiest way is to add entry to dns list? Is there is a certain setup aside from dns wherein all my logs from fortigate,cisco switches and other linux box? Because originally I have this setup wherein I used HAproxy to loadbalance the forwarded logs coming from my linux servers having a splunk forwarder to my two indexers. And I have no problem with it. Now I'm trying to involve my fortigate firewall logs and I'm experiencing this error "ERROR TcpInputProc - Received unexpected 1380997408 byte message (Invalid payload_size=1380997408 received while in parseState=1)!"

Thanks for the help!

0 Karma
Highlighted

Re: Recommended load balancer for indexer

Builder

In my opion the easist way is to use the built in function to loadbalance.

However .. in your case it seems that if you have an working load-balanced syslog solution using HaProxy and syslog-ng / what-ever ... that this might be a good solution and the only thing you need to do is install the splunk forwarder on your cluster nodes (aktive-active or passive-active?) and have them ingest the data/log-files create from syslog.

You then use the built in function in splunk forwarder to load-balance the data into your splunk indexer(-cluster)

The documentation clearly states "dont use 3dje party / hardware loadbalancer" between.

0 Karma
Highlighted

Re: Recommended load balancer for indexer

Builder

which btw is not the same as that it would not work .. just not supported

0 Karma
Highlighted

Re: Recommended load balancer for indexer

Communicator

So this will be may overview according to your comment.

fortigate > splunk forwarder > cluster indexers

Is it?

0 Karma
Highlighted

Re: Recommended load balancer for indexer

Builder

fortigate > SPlunk forwarder > balance on splunk indexers .. will work. You should be able to use the addon on apps.splunk.com to get it going.

or you can use something

fortigate > Syslog-server > file > splunk-forwarder > loadbalance on indexers.

0 Karma
Highlighted

Re: Recommended load balancer for indexer

Communicator

Based on this http://docs.splunk.com/Documentation/Splunk/5.0/Deploy/Configureforwarderswithoutputs.confd#Load_bal...

"The forwarder will load balance between the three receivers listed. If one receiver goes down, the forwarder automatically switches to the next one available."

Somewhat confusing. Why still saying "If one receiver goes down, the forwarder automatically switches to the next one available" if it is load balancing.

0 Karma