Getting Data In

Checksum for seek ptr didn't match, will re-read entire file

akpadhi
Explorer

_TCP_ROUTING = forward_logs
disabled = false
index = 1idx1
sourcetype = LOGS
crcSalt = <SOURCE>

Even though our inputs.conf has crcSalt=<SOURCE>, we see following info messages in splunkd.log and entire log file is getting reindexed for each log entry. Can you please confirm if any other parameters are needed?

11-17-2020 05:07:22.103 -0700 INFO WatchedFile - Checksum for seekptr didn't match, will re-read entire file='Xyz.log'.
11-17-2020 05:07:22.103 -0700 INFO WatchedFile - Will begin reading at offset=0 for file='Xyz.log'.
11-17-2020 05:07:22.104 -0700 WARN CsvLineBreaker - CSV StreamId: 8593577840253621053 has empty line. - data_source="Xyz.log"

 

Labels (3)
0 Karma

somesoni2
Revered Legend

Can you share some sample events from that file? Are they very small?

0 Karma

akpadhi
Explorer
@somesoni2 contents of file below: we didnt specify initcrclength because we are appending to the same file. 11/16/2020 06:37:20 Timestamp test with setting to GMT+5.5 in inputs.conf, MST-06:07:20 11/16/2020 06:46:20 Timestamp test with setting to GMT+5.5 in inputs.conf - test2 , MST-06:16:20 11/16/2020 06:56:20 Timestamp test with setting to GMT+5.5 in inputs.conf - test2 , MST-06:26:20 11/16/2020 19:36:20 Timestamp test with setting to GMT+5.5 in inputs.conf - test2 , MST-07:06:20 11/16/2020 19:40:10 Timestamp test with setting to GMT+5.5 in inputs.conf - test2 , MST-07:10:10 11/16/2020 7:40:15 Timestamp test with setting to GMT+5.5 in inputs.conf - test2 , MST-07:10:15
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try setting the initCrcLength setting to a value higher than 256.  How high depends on how far into the file Splunk has to read to find a change.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Splunk Observability for AI

Don’t miss out on an exciting Tech Talk on Splunk Observability for AI!Discover how Splunk’s agentic AI ...

Splunk Enterprise Security 8.x: The Essential Upgrade for Threat Detection, ...

Watch On Demand the Tech Talk on November 6 at 11AM PT, and empower your SOC to reach new heights! Duration: ...

Splunk Observability as Code: From Zero to Dashboard

For the details on what Self-Service Observability and Observability as Code is, we have some awesome content ...