Hi ! Since I have installed splunk-4.1.2-79191-x64-release as a forwarder on a Windows 64 i'm getting several instances of pairs of Event IDs 55 and 26 that seem to report data corruption.
Here are some examples EVENT ID 55 La structure du système de fichiers (filesystem) sur le disque est endommagée (damaged) et inutilisable (unusable). Exécutez l’utilitaire chkdsk sur le volume C:.
EVENT ID 26 Message de l’application : splunkd.exe - Fichier endommagé : Le fichier (file) ou le répertoire (folder) C:\Program Files\Splunk\var\run\splunk\dispatch\scheduler_nobody_windows_cGVyZm9ybWFuY2Vfc25hcHNob3Q_at_1273975200_1819810544 est endommagé (damaged) et illisible (unreadable). Exécutez (run) l’utilitaire CHKDSK.
NOTE: I've inserted some translations in parenthesis
Does anyone have an idea what is happening?
Thanks in advance
I'm seeing exactly the same problem on a Windows 32 bit server. If I either uninstall Splunk (4.1.2) or disable the Splunk services the disk errors don't happen. As soon as I let Splunk run I get multiple disk errors (event ID 55). I have a mirrored RAID that doesn't report errors under any other situations. I've disabled disk caching but it made no difference.
It is not unusual that Splunk will trigger OS errors that are not easily triggered by other applications, especially with respect to disk, because few other applications write new data continuously at potentially high rates. If the OS error was the one discussed in the other answer here, it is a filesystem level error, and it doesn't really matter what the hardware is.
It seems likely that you have a problem with your disk. This error is generated by the OS/filesystem, not by Splunk itself. Have you in fact run CHKDSK against the volume to see if there are bad sectors?
I think you're right! It's just enoying though that I have to go trough these forced chkdsk at each reboot if I forget to remove the dirty bit from the disks 😞
Thanks again to have taken the time.
Splunk may be more likely to make a filesystem problem show up because it is much more write-intensive (and read-intensive) than almost any other application you are likely to run. I would think that Splunk will not work well if the files are bad, but if the results seem to come back and it's not crashing, I guess you're okay. If it's the error in the forums, it appears that there is no actual corruption of data, so it's basically just a bogus error message from Windows. (Also, the particular Splunk files in question are not permanent data files, but temp files anyway.)
In the mean time, i'd like to know if the problems i'm having are detrimental to splunk or is it programmed to address such issues? Or should I stop using it because it's unreliable in this context?
Or even better, is there a workaround?
Again, thanks to everyone for you precious time. 🙂
Here is an interesting discussion about a know regression in NTFS on Windows 7 that has been recently known to cause similar probs with other products.
Do you think it could be the cause of my errors?
Alas, the problem is only showing up when Splunk is installed. I'm not sure Splunk is actually responsible for the problem. It just happens when it's installed.
Now here's what I did prior my initial post.
- multiple offline check disks without errors. BTW when Event ID 55 is logged, the os flags the filesystem as dirty so I am forced into chkdsk after each reboots.
Restored a disk image of the os prior splunk installation without problems but as soon as I reinst splunk the problems reoccur.
smartmontools runs n all my machines and the hd is in perfect health
thx for your time 🙂