Splunk Search

maxKBps option and limiting a Forwarder's rate of thruput

maverick
Splunk Employee
Splunk Employee

Within the doc page on limits.conf,
http://docs.splunk.com/Documentation/Splunk/latest/Admin/Limitsconf

I found this entry regarding thruput:

[thruput]
maxKBps = <integer>
* If specified and not zero, this limits the speed through the thruput processor to the specified 
rate in kilobytes per second.
* To control the CPU load while indexing, use this to throttle the number of events this indexer 
processes to the rate (in KBps) you specify. 

This description leads me to believe that this specific option will limit the indexer thruput and might not work on the Forwarder to regulate it's thruput rate up to the indexer.

Can someone please confirm this is the case, or (assuming this is not the setting to use) tell me how to regulate the Forwarder's rate of thruput up to the Indexer?

1 Solution

MuS
Legend

Hi maverick

I used this stanza to limit the thruput of Universal Forwarders and Heavy Forwarders towards the indexer. So I can confirm this is the setting you are looking for to regulate the Forwarder's rate of thruput.

cheers

View solution in original post

ChadLangUAB
Path Finder

limits.conf doesn't exist in local. maxKBps = 0 is in limits.conf in default.

4 Indexers

1,322 KB/s
Total Indexing Rate - 331KB/s
Average Indexing Rate - 264 KB/s

jmeager_splunk
Splunk Employee
Splunk Employee

You should never edit the default configurations. It is not found as you need to create one. You should create a new limits.conf within local which will supersede the default folder limits.conf values as required.

yannK
Splunk Employee
Splunk Employee

If in the other hand want to speed your forwarders,
By example because the volume is high, or because the historical logs are large to tail the first time.

You can edit your own local setting to remove all limits.

by example in $SPLUNK_HOME/etc/apps/SplunkUniversalForwarder/local/limits.conf
or on the lightweight forwarder in $SPLUNK_HOME/etc/apps/SplunkLightForwarder/local/limits.conf

[thruput]
# means unlimited
maxKBps = 0

MuS
Legend

Hi maverick

I used this stanza to limit the thruput of Universal Forwarders and Heavy Forwarders towards the indexer. So I can confirm this is the setting you are looking for to regulate the Forwarder's rate of thruput.

cheers

jak3d84
Loves-to-Learn Lots

Hi - can you please paste the solution? The link is not resolving to the stanza.

0 Karma

Jason
Motivator

You'll likely find the setting in etc/apps/SplunkUniversalForwarder/default/limits.conf

kristian_kolb
Ultra Champion

Hi, on a 4.3 UF the .../default/limits.conf states that maxKBps=0 . Does this mean that the throughput is unlimited? I think I remember reading that Light/universal forwarders were capped att 256 KBps transmission rates. Have I missed something?

/k

0 Karma

jbsplunk
Splunk Employee
Splunk Employee

We tested this behavior on a Linux and OSX UF and found that it did work to limit the bandwidth of the UF.

0 Karma
Get Updates on the Splunk Community!

Get ready to show some Splunk Certification swagger at .conf24!

Dive into the deep end of data by earning a Splunk Certification at .conf24. We're enticing you again this ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Now On-Demand Join us to learn more about how you can leverage Service Level Objectives (SLOs) and the new ...

Database Performance Sidebar Panel Now on APM Database Query Performance & Service ...

We’ve streamlined the troubleshooting experience for database-related service issues by adding a database ...