Getting Data In

Checksum for seek ptr didn't match, will re-read entire file

akpadhi
Explorer

_TCP_ROUTING = forward_logs
disabled = false
index = 1idx1
sourcetype = LOGS
crcSalt = <SOURCE>

Even though our inputs.conf has crcSalt=<SOURCE>, we see following info messages in splunkd.log and entire log file is getting reindexed for each log entry. Can you please confirm if any other parameters are needed?

11-17-2020 05:07:22.103 -0700 INFO WatchedFile - Checksum for seekptr didn't match, will re-read entire file='Xyz.log'.
11-17-2020 05:07:22.103 -0700 INFO WatchedFile - Will begin reading at offset=0 for file='Xyz.log'.
11-17-2020 05:07:22.104 -0700 WARN CsvLineBreaker - CSV StreamId: 8593577840253621053 has empty line. - data_source="Xyz.log"

 

Labels (3)
0 Karma

somesoni2
Revered Legend

Can you share some sample events from that file? Are they very small?

0 Karma

akpadhi
Explorer
@somesoni2 contents of file below: we didnt specify initcrclength because we are appending to the same file. 11/16/2020 06:37:20 Timestamp test with setting to GMT+5.5 in inputs.conf, MST-06:07:20 11/16/2020 06:46:20 Timestamp test with setting to GMT+5.5 in inputs.conf - test2 , MST-06:16:20 11/16/2020 06:56:20 Timestamp test with setting to GMT+5.5 in inputs.conf - test2 , MST-06:26:20 11/16/2020 19:36:20 Timestamp test with setting to GMT+5.5 in inputs.conf - test2 , MST-07:06:20 11/16/2020 19:40:10 Timestamp test with setting to GMT+5.5 in inputs.conf - test2 , MST-07:10:10 11/16/2020 7:40:15 Timestamp test with setting to GMT+5.5 in inputs.conf - test2 , MST-07:10:15
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try setting the initCrcLength setting to a value higher than 256.  How high depends on how far into the file Splunk has to read to find a change.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...