Windows Event Logs stop forwarding - why?

[cpenkert]

I have Splunk forwarder installed on many Windows 2008 systems, and recently, the Windows Event logs stopped showing up in my searches, even though forwarding of them is enabled in my /local/inputs.conf

[WinEventLog:Application]
disabled = 0

[WinEventLog:System]
disabled = 0

This just stopped working at 00:00 on 6/1.

I should have also mentioned that we are running 4.05 for the forwarders.

This was working for quite a while and just stopped. I'm unable to get it working again. Any ideas?

Thanks.

[stuartamurray]

I have had this.

Basically the LastAccess time on your log file does NOT necessarily get updated by Windows 2008 installations, if your logging application keeps the file open. Apparently this is an 'optimisation' by Microsoft.

So, in your inputs.conf try the following:

# This flag set for 2008 installations which DONT update LastAccessTime!!
alwaysOpenFile = 1

Be aware that this impacts performance (or so I'm told)

[Mick]

This could be one of a number of reasons, usually related to either a network or a config change of some sort. If it happened on one of your forwarders my first suspicion would be an issue with either that server or the Splunk config on the forwarder instance, if it's affecting all of your forwarders I would suspect the config on your indexer instance to have a problem, or a more general network issue.

We have some general troubleshooting steps you can try documented here

[cpenkert]

I opened a case for this and we are still working on finding out the cause...once I get an answer, I'll post it back here.

[JeremyHagan]

Did you ever get a resolution? I've got the same issue with 2 forwarders out of MANY with identical configuration.

[cpenkert]

all other data from these machines is making it to the indexer just fine. It is only the EventLog entries that aren't getting to the indexer. No resolution in the troubleshooting link that was attached.