Running Splunk 6.0.1 (build 189883), all on Windows-servers, a mix of 2008/2012-servers.
Indexing a lot of SystemOut.log-files from WebSphere, in most cases no problem at all, all events are showing fine, but some of the files are troublesome. For the file mentioned here it is being indexed on and off. Checking now the file was not indexed (or it was indexed, but only the startup-event being logged by WebSphere in that period) until Feb 17'th, then all fine until March the 9'th, and after that nothing, or again to be all precise: Only the startup event of WebSphere every time the JVM is restarted, no other events.
[monitor://E:\logs\*Member*\SystemOut.log] index = klpi sourcetype = websphere:system:out crcSalt = <SOURCE> initCrcLength = 3000
Splunk finds the file without problems
Checking the _internal-index there are no other events for that file than the "File to small...".
WebSphere does rotate the log files when they reach 10Mb, but the date of when the file was rotated does not match the date when Splunk did start/stop receiving events (again - events other than the Startup-message)
It has been a while since I did ask this question, and I realize that I most certainly did solve it when applying timestamp recogniction as described here http://answers.splunk.com/answers/147950/can-i-have-different-timestamp-formats-using-the-same-sourc...
Was the CRC/re-reading issues caused by multiple timestamps in the file? Any idea why that would break things, and do you have any resources to splunk mentioning this?