Heavy forwarder setup - looking at clustering or other HA options

[Aiders1]

Hi all,

I'm new to this forum and found quite a few ideas and solutions to issues admins hit.

The organisation I work for are standing up a new site and requested new pair of heavy forwarders to be installed.

The issue we have been mulling over is how to provide a highly available forwarder cluster at this site.  The forwarders will be based on Linux, will process data from the network (Syslog, netflows etc) and also process files located on a NFS share (service provider managed CIFS/NFS share).

We are using Splunk Cloud but have a deployment server on-prem to manage forwarders on the internal networks.

My question - is there a solution to provide a clustered pair of forwarders that act in an active/passive cluster that allows support for processing files and also accepting network traffic?

cheers

aiders

[PickleRick]

Without using some external solution, you don't have the option to "pair" forwarders and have them monitor the same set of files.

You can monitor them independently from two different forwarders but then you'd obviously have duplicated data.

So a layer of two or more heavy forwarders will give you horizontal scaling and failover capability but this happens _after_ your initial ingestion point (usually UF's).

HF's in this setup are highly available (active-active) but only considering data forwarded from their initial collection point. You can't have "failoverable" inputs on them. It's the outputs logic on the previous layer that does all the work.

[SanjayReddy]

Hi @Aiders1 

I Would recomand  you to  setup heavyforwarders in differnet sites with active--active configuration.

so that if few HFs goes down other can able to accept the data. 

for getting network data , you can enable port on all HF to get data example: enable port 2048 TCP on HF

and provide all HF forwarders IP on syslog config of device.

OR
you can create F5 load balancer IP which contains all the HF IPs, Load balancer can forwarde the data to HF based on avalibilty , you need configure only F5 Load balancer IP on Syslog device 

and on servers where you want to want get the data indtall UF add following configuration which can load balance data between forwarders 

[tcpout]
defaultGroup = HF

[tcpout:HF]
server = forwarder1:9997, forwarder2:9997

you can refer to followinf links

https://docs.splunk.com/Documentation/SplunkCloud/latest/Data/Monitornetworkports
https://docs.splunk.com/Documentation/Splunk/8.2.2/Data/Configureyourinputs 

[Aiders1]

Hi,

Thanks for the details, we are trying to steer away from F5/netscallers for load balancing network traffic and forwarding onto HF's cluster.

thanks

Aiders