File growth rate must be higher than indexing or forwarding rate.

[splunkranger]

03-19-2014 18:47:22.017 -0500 INFO BatchReader - Will retry path="\servername1\dnsdebug\dns.log" after deferring for 10000ms, initCRC changed after being queued (before=0xc3e428fff4486a47, after=0x279979c853e7447a). File growth rate must be higher than indexing or forwarding rate.

I am not able to find any information on this error, can anyone comment?

[dmaislin_splunk]

Might this help?

In $SPLUNK_HOME/etc/apps/SplunkUniversalForwarder/local/limits.conf

[thruput]
maxKBps = 0 # means unlimited

[dmaislin_splunk]

Please check my answer and accept it. Thanks!

[u788332]

I'm seeing same issue but for only one sourcetype , where ~ 10 sourcetypes are specified. I get this for the one sourcetype "File growth rate must be higher than indexing or forwarding rate" , limits.conf is set to 512kbps and is working well for other sourcetypes. The result is a large gaps in indexing for this specific log. However this log is very active many lines per second. Can a bottleneck occur only for high written logs? you would think it would happen on all sourcetypes at the same point in time.. thanks

[dmaislin_splunk]

You can get creative and run another forwarder on different ports on the same system when there is a scenario as you described. It helps eliminate that bottleneck where you can dedicate that forwarder to just that one heavy sourcetype and adjust that specific limits.conf for that.

[splunkranger]

That did solve the issue, just surprised there was no more information available on this error.
Thank you!

[somesoni2]

It seems before Splunk can read the file, its getting changed, especially first few character which Splunk uses for CRC check. Does the program generates the log file overwrites the log file?