2-tier deployment server topology causes forwarders to restart every a few minutes.

[sylim_splunk]

We have a Global deployment server which pushes apps to the Regional deployment servers in AMER, ASIA and EU. Forwarders in each region connect to the Regional DS.
The forwarders appears to be downloading apps successfully but they keep on downloading the same and restart themselves even if there's no changes in apps.

Checksum for apps keeps on changing for the same apps that are supposed to be static. We only do apply updates on the Global DS manually only when it's needed.
Please help.

[sylim_splunk · Accepted solution]

We found the Regional DS errors out with "Application does not exist", which happens the regional DS tries to install the apps and fails after a download. Then Regional DS repeats the Download-install-fail process even if it's successfully downloaded apps for its forwarders. Forwarders keep downloading the apps as the checksums of apps changing each time on the Regional DS - even if there is nothing changed but datetime of files. - This has beenworked around by adding "crossServerChecksum = true" in regonal DS. This stopped the forwarders repeating restarting but still the regional deployment servers repeat "Download-install-fail" this can be controlled by adding "stateOnClient = noop" as it, as a deployment server doesn't need to install and change the state.