Splunk Search

How do you set a default maximum data transfer rate?

jan_wohlers
Path Finder

Dear Splunkers,

is there a maximum KB/s of traffic a forwarder sends to the indexer? I mean is there a limit you can configure? Last week we got some network problems on a dc running splunk. It seems that the network card was too busy to give a quick response.

Thanks for you help!

1 Solution

Ayn
Legend

Yes.

From http://docs.splunk.com/Documentation/Splunk/latest/Deploy/Introducingtheuniversalforwarder

  • "The universal forwarder has a default data transfer rate of 256Kbps"

This can be altered by creating/editing a limits.conf with another value for the maxKbps option. From http://docs.splunk.com/Documentation/Splunk/latest/admin/Limitsconf

[thruput]
maxKBps = <integer>
* If specified and not zero, this limits the speed through the thruput processor to the specified rate in kilobytes per second.
* To control the CPU load while indexing, use this to throttle the number of events this indexer processes to the rate (in KBps) you specify. 

View solution in original post

Ayn
Legend

Yes.

From http://docs.splunk.com/Documentation/Splunk/latest/Deploy/Introducingtheuniversalforwarder

  • "The universal forwarder has a default data transfer rate of 256Kbps"

This can be altered by creating/editing a limits.conf with another value for the maxKbps option. From http://docs.splunk.com/Documentation/Splunk/latest/admin/Limitsconf

[thruput]
maxKBps = <integer>
* If specified and not zero, this limits the speed through the thruput processor to the specified rate in kilobytes per second.
* To control the CPU load while indexing, use this to throttle the number of events this indexer processes to the rate (in KBps) you specify. 

View solution in original post

wdeng
New Member

Where is the best way to find the complete list of config properties for universal forwarder? Do the config files under etc\apps\SplunkUniversalForwarder\default contains the complete set of configurable options?

0 Karma

Ayn
Legend

As it says in the docs, a value of 0 means that no limit is imposed. etc\system\default is far from the only place you could encounter limits.conf however. In the case of Universal Forwarders, many UF-unique settings are set in etc\apps\SplunkUniversalForwarder\default.

You might need to remove or increase the limit for a very busy server like the one you mention. Events will be buffered in memory and on disk according to settings in outputs.conf (http://docs.splunk.com/Documentation/Splunk/latest/admin/outputsconf )

spammenot66
Contributor

@Ayn, thanks for the info.
etc\system\default
etc\apps\SplunkUniversalForwarder\default

0 Karma

jan_wohlers
Path Finder

Thanks for your answer... This means, if in ..etc\system\default\limits.conf the stanza [thruput] maxkbs is set to 0 the default rate of 256kbps is used (if no copy of this conf-file with different values is located in local]?

is 256kbps also best practise and enough for a DC in "bigger" environment with about 60k users? Is there a queue where logs will be put in, when the 256kbps is reached? so this events will be indexed but a bit later?

Thanks for your help!

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!