How much data should be sent to one forwarder?

[nkingsbury]

Hello,
I am setting up a log collector with a Universal Forwarder attached for collecting network logs (syslog-ng) and then sending them to Splunk Cloud.

I am wondering if there is a good rule of thumb/best practice as to how many devices, or how much data should be sent to one collector/forwarder.

I plan to collect logs from: 6 firewalls, 32 routers, 165 switches, as well as some software logs like Cisco ISE.

All of those devices are spread around the world. Should I set up collectors in regional data-centers, or would I be OK sending everything to one?

[woodcock · Accepted solution]

I would do 2 behind a load balancer to give you some fault-tolerance through redundancy. That load can be handled by just one when one of them dies.

[nkingsbury]

Thank you for the answer. It seems to be the common consensus that I should have a load balancer in front of my collectors. Let me spell this out a bit becuase I am quite new to this and I cant find documentation for exactly what I am doing.

So, I will point my network device syslogs at a load balancer, that load balancer is setup to send the traffic to two different syslog-ng/UF's, which then forward the logs up to the cloud indexers.

Is there a recommended load balancer product to use for this case?

[MuS]

Well, don't forget that your load balancer must be HA as well - and yes F5 does a pretty decent job in handling the Splunk traffic.

cheers, MuS

[nkingsbury]

are we talking a physical appliance? I was hopping something like HAProxy or LVS would suffice.

[woodcock]

Yes, that will work, too.

[richgalloway]

A better measure than number of devices is data rate. A UF should have no problem with 256 KB/s or more. IF you're still concerned, stand up multiple syslog-ng servers (with UFs) behind a load balancer.