I have been trying to load balance firewall logs across a 12 node index cluster
the heavy forwarder is under cluster control . It sees all 12 indexes to be able to write to from its "plunk list forward-server". But regardless of all the changes I have been making in the outputs.conf with LB settings it never want to send to more than five when I monitor
| tstats summariesonly=t count WHERE index="network_traffic" by splunk_server _time | timechart span=1m sum(count) by splunk_server
autoLBVolume=1048576
autoLBFrequency=5
I have split the ingest into multiple small files using syslog-ng
Im just wondering is this "five" a hard limit for a forwarder?
or a limitation for the old release I am currently on (will I have to create a horizontal forwarding layer for the firewall logs , artificially splitting the syslog )
Im running Splunk Enterprise 7.3.9
Many thanks , if anyone has any insight
Just to confirm that there is no '5 server' limit for the HF to send data to.
Did you test running splunk btool outputs list on the HF in question to ensure it does list all indexers in the outputs stanza that is in use?
Splunk inputs.conf can also be used to set _TCP_ROUTING, along with props/transforms.conf...assuming you have multiple stanzas in outputs.conf.