Why are we seeing an issue with an EXTREMELY busy forwarder bogging down our indexers?

[ChadLangUAB]

Recently, indexing from that particular forwarder has gotten to be even slower, sometimes falling hours behind. I'm curious as to what the recommendation from the community may be:

1. Configure improved load balancing with props.conf with EVENT_BREAKER_ENABLE setting to true.
2. Changing existing forceTimebasedAutoLB settings to a shorter interval
3. Something else

Our version is 7.0.2

[Jarohnimo]

You'd have to create local/limits.conf and then set It to 0 if you want unlimited. You also may want to consider increasing various queues (parsing queue) if your dealing with a lot of data.

Last you may want to consider increasing the number of pipelines. Get some more firepower In there! Just know it comes at a cost on your remote system (system with the universal forwarder installed).

[ChadLangUAB]

limits.conf doesn't exist in local. maxKBps = 0 is in limits.conf in default.

4 Indexers

1,322 KB/s
Total Indexing Rate - 331 KB/s
Average Indexing Rate - 264 KB/s

[ddrillic]

First we need to determine how much data this forwarder is pushing to the indexers...

[gjanders]

Did you check to make sure the forwarder has not hit the throttle limits? maxKBps as per "maxKBps option and limiting a Forwarder's rate of thruput" or the limits.conf file