If Universal Forwarder crashes, can we throttle the rate at which it sends data to indexer?

[roychen]

Hello,

Assuming that I have a universal forwarder configured to monitor a directory of flat files, e.g. /var/log/, what happens if the following sequence of events happens?

1. Universal forwarder is monitoring files in /var/log
2. Universal forwarder crashes for some reason, or someone accidentally kills the process
3. Files in /var/log are modified, written to, etc. Assume a large number of changes have been made
4. Universal forwarder is restarted

In this situation, will the universal forwarder simply check through /var/log for any modified files, and send all the changes in the logs to the indexer at one go, thus possibly saturating the network bandwidth?

I believe the universal forwarder's max throughput is 256 kb/s, so if there's a large amount of changes, will it always attempt to send data to the indexer at this maximum rate?

Is there any way to throttle the universal forwarder's sending rate?

[gkanapathy · Accepted solution]

The throttled is set to a 256 Kb/s on a UF, but you can set this to whatever rate you like, higher or lower, in the limits.conf file, e.g.:

[thruput]
maxKBps = 128

Setting it to "0" makes the maximum rate unlimited (up to the capacity of the process and the machine).