Splunk Search
Highlighted

maxKBps option and limiting a Forwarder's rate of thruput

Splunk Employee
Splunk Employee

Within the doc page on limits.conf,
http://docs.splunk.com/Documentation/Splunk/latest/Admin/Limitsconf

I found this entry regarding thruput:

[thruput]
maxKBps = <integer>
* If specified and not zero, this limits the speed through the thruput processor to the specified 
rate in kilobytes per second.
* To control the CPU load while indexing, use this to throttle the number of events this indexer 
processes to the rate (in KBps) you specify. 

This description leads me to believe that this specific option will limit the indexer thruput and might not work on the Forwarder to regulate it's thruput rate up to the indexer.

Can someone please confirm this is the case, or (assuming this is not the setting to use) tell me how to regulate the Forwarder's rate of thruput up to the Indexer?

Highlighted

Re: maxKBps option and limiting a Forwarder's rate of thruput

SplunkTrust
SplunkTrust

Hi maverick

I used this stanza to limit the thruput of Universal Forwarders and Heavy Forwarders towards the indexer. So I can confirm this is the setting you are looking for to regulate the Forwarder's rate of thruput.

cheers

View solution in original post

Highlighted

Re: maxKBps option and limiting a Forwarder's rate of thruput

Splunk Employee
Splunk Employee

We tested this behavior on a Linux and OSX UF and found that it did work to limit the bandwidth of the UF.

0 Karma
Highlighted

Re: maxKBps option and limiting a Forwarder's rate of thruput

Ultra Champion

Hi, on a 4.3 UF the .../default/limits.conf states that maxKBps=0 . Does this mean that the throughput is unlimited? I think I remember reading that Light/universal forwarders were capped att 256 KBps transmission rates. Have I missed something?

/k

0 Karma
Highlighted

Re: maxKBps option and limiting a Forwarder's rate of thruput

Motivator

You'll likely find the setting in etc/apps/SplunkUniversalForwarder/default/limits.conf

Highlighted

Re: maxKBps option and limiting a Forwarder's rate of thruput

Splunk Employee
Splunk Employee

If in the other hand want to speed your forwarders,
By example because the volume is high, or because the historical logs are large to tail the first time.

You can edit your own local setting to remove all limits.

by example in $SPLUNK_HOME/etc/apps/SplunkUniversalForwarder/local/limits.conf
or on the lightweight forwarder in $SPLUNK_HOME/etc/apps/SplunkLightForwarder/local/limits.conf

[thruput]
# means unlimited
maxKBps = 0
Highlighted

Re: maxKBps option and limiting a Forwarder's rate of thruput

Path Finder

limits.conf doesn't exist in local. maxKBps = 0 is in limits.conf in default.

4 Indexers

1,322 KB/s
Total Indexing Rate - 331KB/s
Average Indexing Rate - 264 KB/s

Highlighted

Re: maxKBps option and limiting a Forwarder's rate of thruput

Splunk Employee
Splunk Employee

You should never edit the default configurations. It is not found as you need to create one. You should create a new limits.conf within local which will supersede the default folder limits.conf values as required.

Highlighted

Re: maxKBps option and limiting a Forwarder's rate of thruput

SplunkTrust
SplunkTrust