Getting Data In

ERROR TailReader - File will not be read, seekptr checksum did not match

impurush
Contributor

I am getting the below error because of two files has same first two lines including timestamps in the different folder.

ERROR TailReader - File will not be read, seekptr checksum did not match (file=filename.2021-01-19.txt). Last time we saw this initcrc, filename was different. You may wish to use larger initCrcLen for this sourcetype, or a CRC salt on this source. Consult the documentation or file a support case online at http://www.splunk.com/page/submit_issue for more info.

The monitoring stanza has filename.*.txt.

So if I increase the initcrc or crcSalt, then all the files under the folders will get re-indexed.

Along with crcSalt, I tried to use ignoreOlderThan but still, the old files are getting re-indexed.
Example: ignoreOlderThan=1d, means still yesterday files are getting re-indexed.

Any better solution to prevent this?

Labels (1)
0 Karma

impurush
Contributor

I have done few changes to solve the issue.

  • IgnoreOlderthan=1d
  • blacklist=filename.<yesterday>.txt (Because all the older files are in the same folder structure)
  • Restarted the forwarder exactly at 12o clock

Even though few lines of data got re-indexed, however, it is around 10 to 20 lines only which were acceptable.

0 Karma

manjunathmeti
Champion

Don't increase initCrcLength, if the files are in different folders you can set crcSalt=<SOURCE> for which the full directory path to the source file is added to the CRC. This ensures that each file being monitored has a unique CRC. 

impurush
Contributor

Thanks @manjunathmeti  for answering the question. 

However, if we put crcSalt=<SOURCE>, then the older files are getting re-indexed because the files are in same folder.

Example:

/app/folderA/locationA/filename_yyyy-mm-dd.txt
/app/folderB/locationB/filename_yyyy-mm-dd.txt

So if we put crcsalt in any of the file, under the location all the files are getting re-indexed. If we put ignoreOlderthan=1d, still the yesterday file is getting re-indexed.

0 Karma

manjunathmeti
Champion

Forwarder reads the file only if system_current_time - file_modification_time > ignoreOlderthan. Check if yesterdays files are still under this window. 

Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...