Splunk 6 not reading all files in a folder

[somesoni2]

Hi All,

I have created a java program which is generating 20 log files whose names are like this.

l1DebugQueue.log
l2DebugQueue.log
l3DebugQueue.log
....
....
l10DebugQueue.log
l11DebugQueue.log
l12DebugQueue.log
....
....
l20DebugQueue.log
l11DebugQueue.log

I have configured splunk to monitor the folder, but Splunk was not reading any of the file. In _internal index splunkkd logs, I see lot of error messages says

File will not be read, seekptr checksum did not match

Upon googling, I found solution to add following to inputs.conf

 crcSalt = <SOURCE>

Restared splunk after that and now SPlunk is reading 9 files only out of 20 and still seeing the errors in _internal index.

Any idea what can be tried next?

Thanks in advanced.

[rahulroy_splunk · Accepted solution]

The error you mentioned generally happens if first few character of the files matches even though the file names are different. So first check if first few characters (say 100) are same across all files, if yes then, one option would be add a file header for each file which is unique for each file (file name with some timestamp value should do for you). Splunk should be able to read all files after this without using crcSalt.

[somesoni2]

This worked.

[MuS]

did you try btprobe? see this answer http://answers.splunk.com/answers/64679/how-can-i-tell-if-a-file-has-been-recorded-correctly-by-the-...