After a Deployment Server outage, clients all reconnect at once. Any way to stop this?

[Jason]

So our Deployment Server was down for some time, beyond that of the clients' checkin interval, and now that it is back up it is being overwhelmed by hundreds of clients checking in within a few thousanths of a second of each other.

Can anyone think of a way to stop this, without using another tool to sequentially restart splunk on every forwarder?

This is going in as a bug, there's no reason Splunk shouldn't continue to use the checkin interval if unable to connect, or at least backoff and not try every 10-15 seconds.

[dstaulcu]

handshakeRetryIntervalInSecs =
* Defaults to phoneHomeIntervalInSecs [does not seem to be true]
* This sets the handshake retry frequency.
* Could be used to tune the initial connection rate on a new server

http://docs.splunk.com/Documentation/Splunk/6.0.1/admin/Deploymentclientconf