Universal Forwarder failed to start--how do I get it to ingest the missed entries?

[JayX]

The UF service failed to start after a reboot on a Windows Server.   I've addressed that issue, but there are logs that were generated during the downtime that are not being forwarded.  Is there any way to force the entries up?

[richgalloway]

The UF is supposed to pick up where it left off when monitoring files or the Windows event log.  If files were rotated while the UF was down then it's possible the forwarder no longer sees the file.  In that case, you may have to perform a one-shot upload of that file.

[JayX]

That was what I'd assumed would happen, but the forwarder did not pick up any of the events that were logged into the DNS debug log while the forwarder was down.  There are events all the way back to last week in that log, but only the events that have been added since the forwarder was brought back up are being forwarded.

How do I perform a one-shot?

[richgalloway · Accepted solution]

A one-shot upload is for ingesting a file that was skipped entirely and is not suitable for files that were only partially ingested.  You could use it anyway, but you'll end up with duplicate data.

To do a one-shot upload, run splunk add oneshot -source <<filename>> -index foo -sourcetype bar.  Run splunk help add oneshot for details.

[JayX]

Thank you--I may manually extract the gap information and one-shot it from a separate file.

I have also discovered that the forwarder did not fail to start with Windows; it seems to have crashed a couple of days later, so I'm guessing the issue is that the data was still being read by some process on the system, but not being forwarded.