https://community.splunk.com/t5/Getting-Data-In/Universal-Forwarder-and-Offline-Scenarios/m-p/55167#M10747 <P>To guard against loss of data when forwarding to an indexer, you can use Splunk's indexer acknowledgment capability. With indexer acknowledgment, the forwarder will resend any data not acknowledged as "received" by the indexer.</P> <P>This topic is discussed at length in the documentation link below. </P> <P><A href="http://docs.splunk.com/Documentation/Splunk/5.0.2/Deploy/Protectagainstlossofin-flightdata">Protect against loss of in-flight data</A></P> <P>About queues :</P> <P>Memory queues, used by default. A forwarder crashing/shutdown while queueing data will lose the data waiting in the memory queue.</P> <P>Persistent queues, you can configure your forwarders to keep part of the data queued on disk instead of memory, they will be resumed once restarted. </P> <P>see <A href="http://docs.splunk.com/Documentation/Splunk/4.3.1/Data/Usepersistentqueues">http://docs.splunk.com/Documentation/Splunk/4.3.1/Data/Usepersistentqueues</A></P> <P>Also, this Splunk-base answer has some additional details:<BR /> <A href="http://splunk-base.splunk.com/answers/46073/does-the-indexer-acknowledgement-queuelist-persist-across-splunk-restarts">http://splunk-base.splunk.com/answers/46073/does-the-indexer-acknowledgement-queuelist-persist-across-splunk-restarts</A></P> Tue, 04 Jun 2013 23:51:10 GMT okrabbe_splunk 2013-06-04T23:51:10Z https://community.splunk.com/t5/Getting-Data-In/Universal-Forwarder-and-Offline-Scenarios/m-p/55166#M10746 <P>I am using Universal Forwarder on Windows machines to forward events generated by a script.</P> <P><STRONG>Question:</STRONG> What happens if the indexer(s) cannot be reached for some reason? Are the events buffered locally and sent when the connection is restored? Is the behavior in this situation documented anywhere?</P> Tue, 04 Jun 2013 20:59:37 GMT https://community.splunk.com/t5/Getting-Data-In/Universal-Forwarder-and-Offline-Scenarios/m-p/55166#M10746 helge 2013-06-04T20:59:37Z https://community.splunk.com/t5/Getting-Data-In/Universal-Forwarder-and-Offline-Scenarios/m-p/55167#M10747 <P>To guard against loss of data when forwarding to an indexer, you can use Splunk's indexer acknowledgment capability. With indexer acknowledgment, the forwarder will resend any data not acknowledged as "received" by the indexer.</P> <P>This topic is discussed at length in the documentation link below. </P> <P><A href="http://docs.splunk.com/Documentation/Splunk/5.0.2/Deploy/Protectagainstlossofin-flightdata">Protect against loss of in-flight data</A></P> <P>About queues :</P> <P>Memory queues, used by default. A forwarder crashing/shutdown while queueing data will lose the data waiting in the memory queue.</P> <P>Persistent queues, you can configure your forwarders to keep part of the data queued on disk instead of memory, they will be resumed once restarted. </P> <P>see <A href="http://docs.splunk.com/Documentation/Splunk/4.3.1/Data/Usepersistentqueues">http://docs.splunk.com/Documentation/Splunk/4.3.1/Data/Usepersistentqueues</A></P> <P>Also, this Splunk-base answer has some additional details:<BR /> <A href="http://splunk-base.splunk.com/answers/46073/does-the-indexer-acknowledgement-queuelist-persist-across-splunk-restarts">http://splunk-base.splunk.com/answers/46073/does-the-indexer-acknowledgement-queuelist-persist-across-splunk-restarts</A></P> Tue, 04 Jun 2013 23:51:10 GMT https://community.splunk.com/t5/Getting-Data-In/Universal-Forwarder-and-Offline-Scenarios/m-p/55167#M10747 okrabbe_splunk 2013-06-04T23:51:10Z https://community.splunk.com/t5/Getting-Data-In/Universal-Forwarder-and-Offline-Scenarios/m-p/55168#M10748 <P>Thanks, that is helpful. However, the document does not discuss how much data the forwarder still accepts when it cannot reach an indexer (with acks enabled). Will it stop accepting once the queue is full? Or will it accept more? In that case, where is the data stored - in memory, or on disk?</P> Wed, 05 Jun 2013 08:14:32 GMT https://community.splunk.com/t5/Getting-Data-In/Universal-Forwarder-and-Offline-Scenarios/m-p/55168#M10748 helge 2013-06-05T08:14:32Z https://community.splunk.com/t5/Getting-Data-In/Universal-Forwarder-and-Offline-Scenarios/m-p/55169#M10749 <P>helge, I updated the answer to discuss using persistent queues. Hopefully this is what you were looking for.</P> Wed, 05 Jun 2013 13:15:06 GMT https://community.splunk.com/t5/Getting-Data-In/Universal-Forwarder-and-Offline-Scenarios/m-p/55169#M10749 okrabbe_splunk 2013-06-05T13:15:06Z https://community.splunk.com/t5/Getting-Data-In/Universal-Forwarder-and-Offline-Scenarios/m-p/55170#M10750 <P>I think 25MB - <A href="http://docs.splunk.com/Documentation/Splunk/6.4.4/Forwarding/Protectagainstlossofin-flightdata">http://docs.splunk.com/Documentation/Splunk/6.4.4/Forwarding/Protectagainstlossofin-flightdata</A></P> Tue, 07 Mar 2017 16:05:48 GMT https://community.splunk.com/t5/Getting-Data-In/Universal-Forwarder-and-Offline-Scenarios/m-p/55170#M10750 jagadeeshm 2017-03-07T16:05:48Z