Getting Data In

How do forwarders handle rolling logs when an indexer is down?

andrewtrobec
Builder

Hello,

I would like to know how forwarders handle rolling logs when their target indexers become unavailable.  Here is a simple scenario:

  • My application creates a log "application.log"
  • At midnight, "application.log" gets rolled to "application.backup" and a new "application.log" gets created

Assuming my indexer goes down at 11pm and gets restored at 1am the following day, there is 1 hour of log data that will get rolled to "application.backup" and 1 hour of data that is contained in the new "application.log" when the indexer gets restored.

My question relating to the above scenario: will the forwarder keep track of the hour's worth of data that was rolled to "application.backup" as well as the hour's worth of data that is written to "application.log" and send it to the indexer once it becomes available?

Thank you!

Andrew

Labels (2)
Tags (2)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Forwarders buffer their data while they wait for an indexer to become available.  If the data involved does not exceed the forwarder's queue (as set by queueSize and persistentQueueSize in inputs.conf and maxSize in server.conf) then no data will be lost.

---
If this reply helps you, an upvote would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

Forwarders buffer their data while they wait for an indexer to become available.  If the data involved does not exceed the forwarder's queue (as set by queueSize and persistentQueueSize in inputs.conf and maxSize in server.conf) then no data will be lost.

---
If this reply helps you, an upvote would be appreciated.

View solution in original post

andrewtrobec
Builder

Thanks Rich!

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!