topic Re: What more can I do to solve: File too small to check seekcrc, probably truncated. Will re-read entire file in Getting Data In

What more can I do to solve: File too small to check seekcrc, probably truncated. Will re-read entire file

rune_hellem — Thu, 13 Mar 2014 06:28:38 GMT

Running Splunk 6.0.1 (build 189883), all on Windows-servers, a mix of 2008/2012-servers.

Indexing a lot of SystemOut.log-files from WebSphere, in most cases no problem at all, all events are showing fine, but some of the files are troublesome. For the file mentioned here it is being indexed on and off. Checking now the file was not indexed (or it was indexed, but only the startup-event being logged by WebSphere in that period) until Feb 17'th, then all fine until March the 9'th, and after that nothing, or again to be all precise: Only the startup event of WebSphere every time the JVM is restarted, no other events.

From input.confs

[monitor://E:\logs\*Member*\SystemOut.log]
index = klpi
sourcetype = websphere:system:out
crcSalt = <SOURCE>
initCrcLength = 3000

Splunk finds the file without problems

source="E:\\logs\\FondssparingAdminMember01\\SystemOut.log"

Checking the _internal-index there are no other events for that file than the "File to small...".

WebSphere does rotate the log files when they reach 10Mb, but the date of when the file was rotated does not match the date when Splunk did start/stop receiving events (again - events other than the Startup-message)

Re: What more can I do to solve: File too small to check seekcrc, probably truncated. Will re-read entire file

rune_hellem — Mon, 06 Oct 2014 11:06:24 GMT

It has been a while since I did ask this question, and I realize that I most certainly did solve it when applying timestamp recogniction as described here http://answers.splunk.com/answers/147950/can-i-have-different-timestamp-formats-using-the-same-sourcetype.html

Re: What more can I do to solve: File too small to check seekcrc, probably truncated. Will re-read entire file

ljdelight — Wed, 14 Jun 2017 20:47:17 GMT

Was the CRC/re-reading issues caused by multiple timestamps in the file? Any idea why that would break things, and do you have any resources to splunk mentioning this?