topic Re: How do you set a default maximum data transfer rate? in Splunk Search

How do you set a default maximum data transfer rate?

jan_wohlers — Mon, 16 Jul 2012 08:08:10 GMT

Dear Splunkers,

is there a maximum KB/s of traffic a forwarder sends to the indexer? I mean is there a limit you can configure? Last week we got some network problems on a dc running splunk. It seems that the network card was too busy to give a quick response.

Thanks for you help!

Re: How do you set a default maximum data transfer rate?

Ayn — Mon, 16 Jul 2012 08:56:07 GMT

Yes.

From http://docs.splunk.com/Documentation/Splunk/latest/Deploy/Introducingtheuniversalforwarder

"The universal forwarder has a default data transfer rate of 256Kbps"

This can be altered by creating/editing a limits.conf with another value for the maxKbps option. From http://docs.splunk.com/Documentation/Splunk/latest/admin/Limitsconf

[thruput]
maxKBps = <integer>
* If specified and not zero, this limits the speed through the thruput processor to the specified rate in kilobytes per second.
* To control the CPU load while indexing, use this to throttle the number of events this indexer processes to the rate (in KBps) you specify.

Re: How do you set a default maximum data transfer rate?

jan_wohlers — Mon, 16 Jul 2012 09:08:37 GMT

Thanks for your answer... This means, if in ..etc\system\default\limits.conf the stanza [thruput] maxkbs is set to 0 the default rate of 256kbps is used (if no copy of this conf-file with different values is located in local]?

is 256kbps also best practise and enough for a DC in "bigger" environment with about 60k users? Is there a queue where logs will be put in, when the 256kbps is reached? so this events will be indexed but a bit later?

Thanks for your help!

Re: How do you set a default maximum data transfer rate?

Ayn — Mon, 16 Jul 2012 09:42:07 GMT

As it says in the docs, a value of 0 means that no limit is imposed. etc\system\default is far from the only place you could encounter limits.conf however. In the case of Universal Forwarders, many UF-unique settings are set in etc\apps\SplunkUniversalForwarder\default.

You might need to remove or increase the limit for a very busy server like the one you mention. Events will be buffered in memory and on disk according to settings in outputs.conf (http://docs.splunk.com/Documentation/Splunk/latest/admin/outputsconf )

Re: How do you set a default maximum data transfer rate?

spammenot66 — Sun, 05 Mar 2017 15:43:02 GMT

@Ayn, thanks for the info.
etc\system\default
etc\apps\SplunkUniversalForwarder\default

Re: How do you set a default maximum data transfer rate?

wdeng — Wed, 13 Sep 2017 18:22:20 GMT

Where is the best way to find the complete list of config properties for universal forwarder? Do the config files under etc\apps\SplunkUniversalForwarder\default contains the complete set of configurable options?