Is it possible to increase the transfer rate of forwarder in LIMITS.CONF? Any other impacts?

[seema2502]

Dear Splunkers,

I have two forwarders running in my Splunk setup and they are transferring data at a rate of 256 KBPS, which is as per the transfer rate setup in LIMITS.conf.
However this is not serving my purpose, as I can see a delay in the logs being indexed (at least 4 -5 hours). Could you please advice if you have encountered similar issue and what solution could probably fix it,

i did some research of my own from past few days and found that by abruptly increasing the maxKBps = 0 or maxKBps = xxxx, it might result in blocked queues in indexer and forwarder too. Any idea how to get around this ? Will really appreciate any pointer to solve this issue.

thanks in advance!

Thanks,
Seema

[srioux]

I usually crank this up for my busier forwarders, and haven't run into any problems. The real key is knowing what the bottlenecks would be for your particular systems - indexers, as well as forwarders.

For the forwarders, you would want to keep network bandwidth limitations in mind. This includes link speed (100Mbps, GigE, etc?), as well as what else is running over that link (don't want to over-saturate the link and cause problems for your application or whatever's running on the box).

For the indexers, you'll need to keep in mind network bandwidth limitations, as well as disk IO/latency issues as well. For adequately sized indexers, this shouldn't be a problem, but just something else to keep in mind.

You could also have considerations such as WAN link utilization costs, which factor into how much data you'd actually want to send at any given time. Again, this is pretty specific to your (or your organization's) situation.

As for blocked queues, look through the internal logs to see which queues are blocking, and tackle problems from there one-at-a-time. There's a pretty good page on the wiki describing the pipelines:
http://wiki.splunk.com/Community:HowIndexingWorks