In version 4.0, the general consensus is DO NOT USE usePersistentQueue. Further, Splunk has documented that the feature is deprecated. The default behavior (in 4.0) is that, when a forwarder loses its connection to the indexer for whatever reason, it will block continuously instead of writing to disk. For UDP/syslog inputs, this means any events forwarded during the time that queues are blocked will be dropped.
If the concern is prevention of data loss, you should consider going to TCP input and using flat files (e.g. syslog-ng).
Be forewarned that a usePersistentQueue-enabled configuration will require very careful planning as it is heavily dependent on hardware resources. You will be responsible for fully testing this configuration before Production implementation. If you still choose to go this route, be aware of the scenario where, if queues are blocked and the Splunk forwarder is writing to disk, you cannot restart the forwarder until the "issue" is resolved or events will be lost.
... View more