What does "Ignoring path due to: Could not checksum file" mean?

[williamche]

In the splunkd.log file on my Light Forwarder server I have been seeing these errors for all the DHCP log files that I'm monitoring:

ERROR TailingProcessor - Ignoring path due to: Could not checksum file='\\wins1\dhcp$\DhcpSrvLog.Fri'.
ERROR TailingProcessor - Ignoring path due to: Could not checksum file='\\wins1\dhcp$\DhcpSrvLog.Mon'.
ERROR TailingProcessor - Ignoring path due to: Could not checksum file='\\wins1\dhcp$\DhcpSrvLog.Sat'.
ERROR TailingProcessor - Ignoring path due to: Could not checksum file='\\wins1\dhcp$\DhcpSrvLog.Sun'.
ERROR TailingProcessor - Ignoring path due to: Could not checksum file='\\wins1\dhcp$\DhcpSrvLog.Thu'.
ERROR TailingProcessor - Ignoring path due to: Could not checksum file='\\wins1\dhcp$\DhcpSrvLog.Tue'.
ERROR TailingProcessor - Ignoring path due to: Could not checksum file='\\wins1\dhcp$\DhcpSrvLog.Wed'.

These errors first showed up about a month ago and Splunk has not indexed any new DHCP events since. These are log files created by Windows 2003's DHCP service. There are 7 files on each DHCP server, one for each day of the week. These are also very small files of maybe 20 to 30 KB each. The first 24 lines of each files contain a list of all the DHCP IDs and their descriptions. Because of the file size and format I used crcSalt = <SOURCE> to help with the ingest.

The Splunk Light Forwarder service is running as a domain user account with read-only access to these files. There is no connectivity issues between the Windows machines, and I have no problems reading these files using the same user account.

Here is how the monitor statement is setup in input.conf:

[monitor://\\wins1\dhcp$]
disabled = 0
followTail = 0
host = WINS1
index = dhcpd
sourcetype = DhcpSrvLog
crcSalt = <SOURCE>
# whitelist = \\DhcpSrvLog\.(Sun|Mon|Tue|Wed|Thu|Fri|Sat)$
whitelist = DhcpSrvLog

I upgraded both the Index and LF servers to version 4.2 and, so far, have not fixed the problem. Has anyone else come across these "Could not checksum file" errors? Any suggestions on how to go about troubleshooting this?

Thanks.

[peetchow]

i have the same issue and the DHCP servers i have that are producing the logs with the missing comma are Windows Server 2000. my splunk build is version 4.3, build 115073.

I also have 2 DHCP servers that are Windows Server 2003 and there logs are fine.

How can i go and add the comma to the 2000 boxes?

[williamche · Accepted solution]

Issue turned out to be a bug that has been addressed in Splunk 4.2.2. The SPL number is SPL-39103 "UF 4.2 Windows2003 crash, causing files to be ignored since for checksum"

In laymen terms, I was told that, the following line written by the Windows 2003 DHCP server at the beginning of each log file has a missing comma that caused Splunk to throw a checksum error:

ID Date,Time,Description,IP Address,Host Name,MAC Address

There should have been a comma after "ID"

Upgrading to Splunk 4.2.2 resolved the problem. Thanks yanniK, amrit and araitz for your inputs!

[yannK]

This is because you have a "$" in your path, this will not work.
please escape it or change the path to your monitored file.

[yannK]

please disregard, it seems that the $ is not an issue.

[williamche]

added category.FileInputTracker=DEBUG to etc/log.cfg revealed the following new log traces for each of the Win DHCP log files:

04-18-2011 16:56:55.321 -0400 DEBUG FileInputTracker - File seems to be a MS DHCP log - trying to skip header.

04-18-2011 16:56:55.321 -0400 INFO FileInputTracker - Not enough data (in 10196 bytes) to compute MS DHCP CRC.

04-18-2011 16:56:55.321 -0400 ERROR TailingProcessor - Ignoring path due to: Could not checksum file='\wins2\dhcp$\DhcpSrvLog.Sun'.

Tried changing the CHECK_METHOD for these log files to entire_md5 did not make any difference. Is my DHCP servers not getting enough hit to generate a large enough sample for Splunk to ingest?

In any case I did create a case and I will append this new bit of information. Thanks for the suggestion, amrit!

-w

[amrit]

Can you also upload the log file in question to the case? Thanks.

[amrit]

This should only happen when there's an actual file access error. For example, a failing disk, or in your case, perhaps file permission issues.

It's possible that putting FileInputTracker (etc/log.cfg) in INFO or DEBUG may reveal more clues, but the above scenario is most likely.

[araitz]

I have not seen this issue before. Have you opened a support case?