Most efficent way to ignore last line when monitoring to avoid issues with external log buffering?

[JSkier]

I'm having an issue with a custom application log file (text, xml, single line) where the log buffering done by the custom app is causing an occasional flurry of random splits when ingested by splunk. Basically, the last line is buffered (not completely written to immediately) by the application, for performance, and absolutely may not be changed. So I'd like the forwarder to ignore the very last line, until it is no longer the last line. Basically, tail the log file down to the second to last line in the log file monitored.

I am running splunk enterprise 6.1 (splunk on Linux), and a Windows application server with splunk forwarder.

[splunkIT]

Perhaps you would like to try this solution:

http://answers.splunk.com/answers/100006/what-is-the-expected-behavior-for-slow-writing-log-files.ht...

[JSkier]

I did notice that, and have been testing it out. There is some improvement but it doesn't fix the problem yet. I'm at time_before_close = 300 presently. I'll try testing it more with higher values and post any findings here.

The buffering is for performance reasons, and not something I could have changed unfortunately.

Was really hoping for an easy solution, there does not appear to be one. Support recommended a daily batch route, which would mean being a day behind with these logs.