Getting Data In

Why are we seeing an issue with an EXTREMELY busy forwarder bogging down our indexers?

ChadLangUAB
Path Finder

Recently, indexing from that particular forwarder has gotten to be even slower, sometimes falling hours behind. I'm curious as to what the recommendation from the community may be:

  1. Configure improved load balancing with props.conf with EVENT_BREAKER_ENABLE setting to true.
  2. Changing existing forceTimebasedAutoLB settings to a shorter interval
  3. Something else

Our version is 7.0.2

0 Karma

Jarohnimo
Builder

You'd have to create local/limits.conf and then set It to 0 if you want unlimited. You also may want to consider increasing various queues (parsing queue) if your dealing with a lot of data.

Last you may want to consider increasing the number of pipelines. Get some more firepower In there! Just know it comes at a cost on your remote system (system with the universal forwarder installed).

0 Karma

ChadLangUAB
Path Finder

limits.conf doesn't exist in local. maxKBps = 0 is in limits.conf in default.

4 Indexers

1,322 KB/s
Total Indexing Rate - 331 KB/s
Average Indexing Rate - 264 KB/s

0 Karma

ddrillic
Ultra Champion

First we need to determine how much data this forwarder is pushing to the indexers...

0 Karma

gjanders
SplunkTrust
SplunkTrust

Did you check to make sure the forwarder has not hit the throttle limits? maxKBps as per "maxKBps option and limiting a Forwarder's rate of thruput" or the limits.conf file

0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!