Getting Data In

Checksum for seek ptr didn't match, will re-read entire file

akpadhi
Explorer

_TCP_ROUTING = forward_logs
disabled = false
index = 1idx1
sourcetype = LOGS
crcSalt = <SOURCE>

Even though our inputs.conf has crcSalt=<SOURCE>, we see following info messages in splunkd.log and entire log file is getting reindexed for each log entry. Can you please confirm if any other parameters are needed?

11-17-2020 05:07:22.103 -0700 INFO WatchedFile - Checksum for seekptr didn't match, will re-read entire file='Xyz.log'.
11-17-2020 05:07:22.103 -0700 INFO WatchedFile - Will begin reading at offset=0 for file='Xyz.log'.
11-17-2020 05:07:22.104 -0700 WARN CsvLineBreaker - CSV StreamId: 8593577840253621053 has empty line. - data_source="Xyz.log"

 

Labels (3)
0 Karma

somesoni2
Revered Legend

Can you share some sample events from that file? Are they very small?

0 Karma

akpadhi
Explorer
@somesoni2 contents of file below: we didnt specify initcrclength because we are appending to the same file. 11/16/2020 06:37:20 Timestamp test with setting to GMT+5.5 in inputs.conf, MST-06:07:20 11/16/2020 06:46:20 Timestamp test with setting to GMT+5.5 in inputs.conf - test2 , MST-06:16:20 11/16/2020 06:56:20 Timestamp test with setting to GMT+5.5 in inputs.conf - test2 , MST-06:26:20 11/16/2020 19:36:20 Timestamp test with setting to GMT+5.5 in inputs.conf - test2 , MST-07:06:20 11/16/2020 19:40:10 Timestamp test with setting to GMT+5.5 in inputs.conf - test2 , MST-07:10:10 11/16/2020 7:40:15 Timestamp test with setting to GMT+5.5 in inputs.conf - test2 , MST-07:10:15
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try setting the initCrcLength setting to a value higher than 256.  How high depends on how far into the file Splunk has to read to find a change.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Building Reliable Asset and Identity Frameworks in Splunk ES

 Accurate asset and identity resolution is the backbone of security operations. Without it, alerts are ...

Cloud Monitoring Console - Unlocking Greater Visibility in SVC Usage Reporting

For Splunk Cloud customers, understanding and optimizing Splunk Virtual Compute (SVC) usage and resource ...

Automatic Discovery Part 3: Practical Use Cases

If you’ve enabled Automatic Discovery in your install of the Splunk Distribution of the OpenTelemetry ...