Monitoring Splunk

Splunk 6 not reading all files in a folder

somesoni2
Revered Legend

Hi All,

I have created a java program which is generating 20 log files whose names are like this.

l1DebugQueue.log
l2DebugQueue.log
l3DebugQueue.log
....
....
l10DebugQueue.log
l11DebugQueue.log
l12DebugQueue.log
....
....
l20DebugQueue.log
l11DebugQueue.log

I have configured splunk to monitor the folder, but Splunk was not reading any of the file. In _internal index splunkkd logs, I see lot of error messages says

File will not be read, seekptr checksum did not match

Upon googling, I found solution to add following to inputs.conf

 crcSalt = <SOURCE>

Restared splunk after that and now SPlunk is reading 9 files only out of 20 and still seeing the errors in _internal index.

Any idea what can be tried next?

Thanks in advanced.

Tags (2)
0 Karma
1 Solution

rahulroy_splunk
Path Finder

The error you mentioned generally happens if first few character of the files matches even though the file names are different. So first check if first few characters (say 100) are same across all files, if yes then, one option would be add a file header for each file which is unique for each file (file name with some timestamp value should do for you). Splunk should be able to read all files after this without using crcSalt.

View solution in original post

rahulroy_splunk
Path Finder

The error you mentioned generally happens if first few character of the files matches even though the file names are different. So first check if first few characters (say 100) are same across all files, if yes then, one option would be add a file header for each file which is unique for each file (file name with some timestamp value should do for you). Splunk should be able to read all files after this without using crcSalt.

View solution in original post

somesoni2
Revered Legend

This worked.

0 Karma

MuS
SplunkTrust
SplunkTrust
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.