What versions are at play?
In 4.0.x, a transient EOF (that splunk manages to reach) mid-event would generate a done key, which would shut down all the machinery dealing with the datastream, ensureing that the event cuts at that point. This wasn't ideal, but since this pattern of writes is uncommin, and since splunk's tailing machinery often wasn't on top of all files at all time, it wasn't that large a problem.
In 4.1.x, our tailing machinery is much much much more low latency, and so the strategy changed to wait for EOF to stay still for a few seconds before shutting down the machinery, so generally speaking should have this problem less often.
However, if you are getting this problem it's likely that part of your text is sitting in a application or libc buffer for a significant amount of time, possibly during a brief lull. The buffers can be of significant size (several kilobytes).
I think you just have to adopt the same behavior that most logging applications do: that of flushing at the end of your events. It's a good practice in general. If your app says "omg, huge bug happening, dying." and then crashes when that data is still in the buffers, you'll never see the message. Flushing should not have a significant cost, it's just the matter of handing that memory buffer from your app to the OS, not the matter of waiting for the write to complete.
... View more
