topic Re: Why are universal forwarders reporting "File will not be read, seekptr checksum did not match"? in Getting Data In

Why are universal forwarders reporting "File will not be read, seekptr checksum did not match"?

prakash007 — Wed, 04 May 2016 14:48:37 GMT

We're getting bunch of these exceptions on our Universal Forwarders...any help would be appreciated and I can provide more info if needed...

1) ERROR TailReader -File will not be read, seekptr checksum did not matchit says file will not be read. Does that mean it's ignoring the live log which is logged in the path specified..?

ERROR TailReader -File will not be read, seekptr checksum did not match (file=/opt/app/ws/server/kv_JVM01/log/responseTime.2016-05-04.log).  Last time we saw this initcrc, filename was different.  You may wish to use larger initCrcLen for this sourcetype, or a CRC salt on this source.  Consult the documentation or file a support case online at http://www.splunk.com/page/submit_issue for more info.

ERROR TailReader -File will not be read, seekptr checksum did not match (file=/opt/app/ws/server/jr_LCMI/log/server.log).  Last time we saw this initcrc, filename was different.  You may wish to use larger initCrcLen for this sourcetype, or a CRC salt on this source.  Consult the documentation or file a support case online at http://www.splunk.com/page/submit_issue for more info

2) INFO WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file Does that mean it's re-indexing entire file again..?

INFO  WatchedFile - File too small to check seekcrc, probably truncated.  Will re-read entire file='/opt/app/ws/ATG-Data/home/servers/ku_JVM00/logs/apps.log'

 INFO  WatchedFile - File too small to check seekcrc, probably truncated.  Will re-read entire file='/opt/app/ws/server/ra_JVM00/log/server.log'.

As soon as I restart UFs, I see the message below. What do the offset numbers mean..?

INFO  WatchedFile - Will begin reading at offset=0 for file='file=/opt/app/ws/server/kv_JVM01/log/responseTime.2016-05-04.log'.
INFO  WatchedFile - Will begin reading at offset=90 for file='/opt/app/ws/server/ra_JVM00/log/server.log'.
INFO  WatchedFile - Will begin reading at offset=180 for file='file=/opt/app/ws/server/kv_JVM00/log/responseTime.2016-05-04.log'.

Configs on my universal forwarders:

inputs.conf

[monitor:///opt/app/ws/server/*/log/server.log]
sourcetype=log4j
index=testenv

[monitor:///opt/app/ws/server/*/log/responseTime.*.log]
Sourcetype=responseTime
index=testenv

[monitor:///opt/app/ws/ATG-Data/home/servers/*/logs/apps.log]
Sourcetype=apps
index=testenv

outputs.conf

#conpressed and useACK were not set for some of the UFs
#splhfserver is a HF which route the data to Splunk, it's not indexing locally..

[tcpout]
defaultGroup = splhfdataforwarder
compressed = true 
useACK = true

[tcpout:splhfdatafowarder]
Server=splhfserver:9997

Re: Why are universal forwarders reporting "File will not be read, seekptr checksum did not match"?

ddrillic — Wed, 04 May 2016 19:03:13 GMT

ERROR TailingProcessor - Ignoring path

speaks about it.

Re: Why are universal forwarders reporting "File will not be read, seekptr checksum did not match"?

prakash007 — Wed, 04 May 2016 19:13:06 GMT

ddrilic,

If i use a crcSalt on all there sources, is there any chance of re-indexing the data. The answers seems to be relevant to v4.1.5 of splunk.

Re: Why are universal forwarders reporting "File will not be read, seekptr checksum did not match"?

ddrillic — Wed, 04 May 2016 20:19:27 GMT

it's applicable to 6.4 - 6.4 - Inputsconf

-- If i use a crcSalt on all there sources, is there any chance of re-indexing the data.
Depends, I guess, on the value of initCrcLength. If it's, let's say, 2000, instead of the default 256, you probably should be ok.

Re: Why are universal forwarders reporting "File will not be read, seekptr checksum did not match"?

prakash007 — Tue, 29 Sep 2020 09:36:10 GMT

Actually i'm having hard time in figuring it out....

The actual path of the monitor stanza would include..

[monitor:///opt/app/ws/server/*/log/server.log]

/opt/app/ws/server/ra_JVM00/log/server.log
/opt/app/ws/server/ra_JVM01/log/server.log
/opt/app/ws/server/pr_INS00/log/server.log
/opt/app/ws/server/pr_INS02/log/server.log

Can you suggest me which would the best option to go with...

     [monitor:///opt/app/ws/server/*/log/server.log]
     sourcetype=log4j
     crcSalt = <SOURCE>
     index=testenv

     [monitor:///opt/app/ws/server/*/log/server.log]
     sourcetype=log4j
     initCrcLength = 2000
     index=testenv

Re: Why are universal forwarders reporting "File will not be read, seekptr checksum did not match"?

somesoni2 — Wed, 04 May 2016 21:28:33 GMT

Option 2, avoids chances of duplication.

Re: Why are universal forwarders reporting "File will not be read, seekptr checksum did not match"?

prakash007 — Wed, 04 May 2016 21:39:25 GMT

somesoni,
If I go with option 2, will it be a resolution for both of these issues...

1.ERROR TailReader -File will not be read, seekptr checksum did not match
2.INFO WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file

option 2

 [monitor:///opt/app/ws/server/*/log/server.log]
 sourcetype=log4j
 initCrcLength = 2000
 index=testenv

If possible can you explain a bit on this offset numbers..?

INFO  WatchedFile - Will begin reading at offset=0 for file='file=/opt/app/ws/server/kv_JVM01/log/responseTime.2016-05-04.log'.
 INFO  WatchedFile - Will begin reading at offset=90 for file='/opt/app/ws/server/ra_JVM00/log/server.log'.
 INFO  WatchedFile - Will begin reading at offset=180 for file='file=/opt/app/ws/server/kv_JVM00/log/responseTime.2016-05-04.log'

Re: Why are universal forwarders reporting "File will not be read, seekptr checksum did not match"?

ddrillic — Thu, 05 May 2016 14:21:35 GMT

We ended up doing something like -

      [monitor:///opt/app/ws/server/*/log/server.log]
      sourcetype=log4j
      crcSalt = <source>
      initCrcLength = 2000
      index=testenv

Re: Why are universal forwarders reporting "File will not be read, seekptr checksum did not match"?

harsmarvania57 — Thu, 05 May 2016 14:40:34 GMT

If your log files presents on NFS file system then you might be hitting this issue for reindexing logs https://answers.splunk.com/answers/130729/splunk-reindexing-files-when-using-remote-shared-filesystem.html

Re: Why are universal forwarders reporting "File will not be read, seekptr checksum did not match"?

prakash007 — Thu, 05 May 2016 14:45:05 GMT

Will go with the above said option, is there any ways to check if logs are re-indexed or double indexed...?

I did follow this post, but the numbers do not match...

https://answers.splunk.com/answers/24588/how-can-i-check-that-splunk-indexed-the-entire-contents-of-a-given-file.html

Re: Why are universal forwarders reporting "File will not be read, seekptr checksum did not match"?

prakash007 — Thu, 05 May 2016 21:09:47 GMT

when i add the above settings, i still see this messages on splunkd.logs, how did you resolve this...?

File too small to check seekcrc, probably truncated.  Will re-read entire file=...filepath
File too small to check seekcrc, probably truncated.  Will re-read entire file=...filepath
05-05-2016 16:09:54.601 -0500 INFO  WatchedFile - Logfile truncated while open, original pathname file='/opt/app/ws/server/ra_JVM00/log/server.log'., will begin reading from start.
05-05-2016 16:09:54.602 -0500 INFO  WatchedFile - Logfile truncated while open, original pathname ffile='/opt/app/ws/server/ra_JVM04/log/server.log'., will begin reading from start.
05-05-2016 16:09:54.605 -0500 INFO  WatchedFile - File too small to check seekcrc, probably truncated.  Will re-read entire file='/opt/app/ws/server/ra_JVM02/log/server.log'.

Re: Why are universal forwarders reporting "File will not be read, seekptr checksum did not match"?

ddrillic — Fri, 06 May 2016 20:32:26 GMT

So how big is this file? - /opt/app/ws/server/ra_JVM02/log/server.log

Re: Why are universal forwarders reporting "File will not be read, seekptr checksum did not match"?

prakash007 — Fri, 06 May 2016 21:10:52 GMT

this is what i see from the path /opt/app/ws/server/ra_JVM02/log/server.log

$ cat server.log | wc -l

1205
$ cat server.log | wc -c
236896
Adding a crcSalt helped me to get rid of this messages...

 ERROR TailReader -File will not be read, seekptr checksum did not match (file=/opt/app/ws/server/kv_JVM01/log/responseTime.2016-05-04.log).  Last time we saw this initcrc, filename was different.  You may wish to use larger initCrcLen for this sourcetype, or a CRC salt on this source.  Consult the documentation or file a support case online at http://www.splunk.com/page/submit_issue for more info.

Adding a initCrcLength = 2000 OR followTail =1, doesn't help me with this issue...not sure how i can verify if this message is a sign of re-indexing or not..?

File too small to check seekcrc, probably truncated.  Will re-read entire file=...filepath
File too small to check seekcrc, probably truncated.  Will re-read entire file=...filepath
05-05-2016 16:09:54.601 -0500 INFO  WatchedFile - Logfile truncated while open, original pathname file='/opt/app/ws/server/ra_JVM00/log/server.log'., will begin reading from start.
 05-05-2016 16:09:54.602 -0500 INFO  WatchedFile - Logfile truncated while open, original pathname ffile='/opt/app/ws/server/ra_JVM04/log/server.log'., will begin reading from start.
 05-05-2016 16:09:54.605 -0500 INFO  WatchedFile - File too small to check seekcrc, probably truncated.  Will re-read entire file='/opt/app/ws/server/ra_JVM02/log/server.log'.

Re: Why are universal forwarders reporting "File will not be read, seekptr checksum did not match"?

Gayatri — Wed, 23 Oct 2024 13:01:01 GMT

Here <source>means have you typed exact path of the file or just <source> as is?