Hello,
I am setting up a log collector with a Universal Forwarder attached for collecting network logs (syslog-ng) and then sending them to Splunk Cloud.
I am wondering if there is a good rule of thumb/best practice as to how many devices, or how much data should be sent to one collector/forwarder.
I plan to collect logs from: 6 firewalls, 32 routers, 165 switches, as well as some software logs like Cisco ISE.
All of those devices are spread around the world. Should I set up collectors in regional data-centers, or would I be OK sending everything to one?
I would do 2 behind a load balancer to give you some fault-tolerance through redundancy. That load can be handled by just one when one of them dies.
I would do 2 behind a load balancer to give you some fault-tolerance through redundancy. That load can be handled by just one when one of them dies.
Thank you for the answer. It seems to be the common consensus that I should have a load balancer in front of my collectors. Let me spell this out a bit becuase I am quite new to this and I cant find documentation for exactly what I am doing.
So, I will point my network device syslogs at a load balancer, that load balancer is setup to send the traffic to two different syslog-ng/UF's, which then forward the logs up to the cloud indexers.
Is there a recommended load balancer product to use for this case?
Well, don't forget that your load balancer must be HA as well - and yes F5 does a pretty decent job in handling the Splunk traffic.
cheers, MuS
are we talking a physical appliance? I was hopping something like HAProxy or LVS would suffice.
Yes, that will work, too.
A better measure than number of devices is data rate. A UF should have no problem with 256 KB/s or more. IF you're still concerned, stand up multiple syslog-ng servers (with UFs) behind a load balancer.