topic Re: Repeated ntfs problems on Windows 7 64 in Getting Data In

Repeated ntfs problems on Windows 7 64

MrSplunksta — Sun, 16 May 2010 09:20:23 GMT

Hi ! Since I have installed splunk-4.1.2-79191-x64-release as a forwarder on a Windows 64 i'm getting several instances of pairs of Event IDs 55 and 26 that seem to report data corruption.

Here are some examples EVENT ID 55 La structure du système de fichiers (filesystem) sur le disque est endommagée (damaged) et inutilisable (unusable). Exécutez l’utilitaire chkdsk sur le volume C:.

EVENT ID 26 Message de l’application : splunkd.exe - Fichier endommagé : Le fichier (file) ou le répertoire (folder) C:\Program Files\Splunk\var\run\splunk\dispatch\scheduler_nobody_windows_cGVyZm9ybWFuY2Vfc25hcHNob3Q_at_1273975200_1819810544 est endommagé (damaged) et illisible (unreadable). Exécutez (run) l’utilitaire CHKDSK.

NOTE: I've inserted some translations in parenthesis

Does anyone have an idea what is happening?

Thanks in advance

Georges

Re: Repeated ntfs problems on Windows 7 64

gkanapathy — Sun, 16 May 2010 13:53:42 GMT

It seems likely that you have a problem with your disk. This error is generated by the OS/filesystem, not by Splunk itself. Have you in fact run CHKDSK against the volume to see if there are bad sectors?

Re: Repeated ntfs problems on Windows 7 64

MrSplunksta — Mon, 17 May 2010 01:32:48 GMT

Alas, the problem is only showing up when Splunk is installed. I'm not sure Splunk is actually responsible for the problem. It just happens when it's installed.

Now here's what I did prior my initial post.
- multiple offline check disks without errors. BTW when Event ID 55 is logged, the os flags the filesystem as dirty so I am forced into chkdsk after each reboots.

Restored a disk image of the os prior splunk installation without problems but as soon as I reinst splunk the problems reoccur.
smartmontools runs n all my machines and the hd is in perfect health

thx for your time 🙂

Re: Repeated ntfs problems on Windows 7 64

MrSplunksta — Mon, 17 May 2010 01:35:13 GMT

Here is an interesting discussion about a know regression in NTFS on Windows 7 that has been recently known to cause similar probs with other products.

http://social.technet.microsoft.com/Forums/en/w7itprogeneral/thread/df935a52-a0a9-4f67-ac82-bc39e0585148

Do you think it could be the cause of my errors?

Re: Repeated ntfs problems on Windows 7 64

MrSplunksta — Mon, 17 May 2010 01:37:14 GMT

In the mean time, i'd like to know if the problems i'm having are detrimental to splunk or is it programmed to address such issues? Or should I stop using it because it's unreliable in this context?

Or even better, is there a workaround?

Again, thanks to everyone for you precious time. 🙂

Re: Repeated ntfs problems on Windows 7 64

gkanapathy — Mon, 17 May 2010 16:11:56 GMT

Splunk may be more likely to make a filesystem problem show up because it is much more write-intensive (and read-intensive) than almost any other application you are likely to run. I would think that Splunk will not work well if the files are bad, but if the results seem to come back and it's not crashing, I guess you're okay. If it's the error in the forums, it appears that there is no actual corruption of data, so it's basically just a bogus error message from Windows. (Also, the particular Splunk files in question are not permanent data files, but temp files anyway.)

Re: Repeated ntfs problems on Windows 7 64

MrSplunksta — Tue, 18 May 2010 10:56:19 GMT

I think you're right! It's just enoying though that I have to go trough these forced chkdsk at each reboot if I forget to remove the dirty bit from the disks 😞

Thanks again to have taken the time.

🙂

Re: Repeated ntfs problems on Windows 7 64

slever — Fri, 04 Jun 2010 15:04:12 GMT

I'm seeing exactly the same problem on a Windows 32 bit server. If I either uninstall Splunk (4.1.2) or disable the Splunk services the disk errors don't happen. As soon as I let Splunk run I get multiple disk errors (event ID 55). I have a mirrored RAID that doesn't report errors under any other situations. I've disabled disk caching but it made no difference.

Re: Repeated ntfs problems on Windows 7 64

MrSplunksta — Wed, 09 Jun 2010 01:12:43 GMT

did you try 4.1.3 to see if it resolves this problem. I'll be upgrading my self later this week.

Re: Repeated ntfs problems on Windows 7 64

slever — Thu, 10 Jun 2010 14:58:06 GMT

Thanks I'll try the latest version and see if that fixes it.

Re: Repeated ntfs problems on Windows 7 64

gkanapathy — Fri, 16 Jul 2010 20:59:06 GMT

It is not unusual that Splunk will trigger OS errors that are not easily triggered by other applications, especially with respect to disk, because few other applications write new data continuously at potentially high rates. If the OS error was the one discussed in the other answer here, it is a filesystem level error, and it doesn't really matter what the hardware is.

Re: Repeated ntfs problems on Windows 7 64

MrSplunksta — Fri, 23 Jul 2010 00:55:15 GMT

I can confirm that the problem has completely vanished once I upgraded to 4.1.4 😉

Re: Repeated ntfs problems on Windows 7 64

RIADH — Wed, 15 Feb 2012 13:58:35 GMT

Hi, all members,
I'm seeing exactly the same problem on a Windows XP 32 bit.

Re: Repeated ntfs problems on Windows 7 64

Brian_Osburn — Wed, 15 Feb 2012 14:13:56 GMT

I'd suggest opening another question regarding this since this was from almost 2 years ago, and the latest version is now 4.3..

Brian