topic Re: WinEventLog:* on Windows 2008 and Splunk 4.3.2 forwarders - Splunk could not get the description for this event. in Getting Data In

WinEventLog:* on Windows 2008 and Splunk 4.3.2 forwarders - Splunk could not get the description for this event.

jeff — Tue, 08 May 2012 17:27:26 GMT

We have Universal Forwarders installed on Windows 2003 & 2008 Servers, plus a heavy forwarder on Windows 2008...

We updated to 4.3.2 on all forwarders in April, and converted all but one system configured as heavy forwarders to universal forwarders. Most of the systems were previously running 4.2.4 heavy forwarders, though a few were running 4.3.1 Universal Forwarders.

Last week I noticed, 11 of my 15 Windows forwarders displayed the "Splunk could not get the description for this event" message in 4,647 events for a 24 hour period, excluding domain controller security logs (in which case it goes into the millions). In the case of the domain controller, cycling the SplunkForwarder service once or twice usually clears up the messages from the WinEventLog:Security, though I'll continue to get the error message on the DCs in the Application and System Logs.

05/08/2012 01:19:29 PM LogName=System SourceName=Service Control Manager EventCode=7040 EventType=4 ComputerName=DC2.hersheymed.net User=SYSTEM Sid=S-1-5-18 SidType=1 TaskCategory=None OpCode=None RecordNumber=211980 Keywords=None Message=Splunk could not get the description for this event. Either the component that raises this event is not installed on your local computer or the installation is corrupt. FormatMessage error: The handle is invalid. Got the following information from this event: Windows Modules Installer demand start auto start TrustedInstaller

All 11 are Windows 2008(32-bit, 64-bit, and R2), the other four are all Windows 2003. The number of messages in the System and Application logs that display this behavior far exceeds the number of messages that do not. Indexes are 4.3.2 on RedHat, in case it matters. There are no (or very few if they're buried in the data) events with this behavior prior to updating the forwarder on any given host.

I've had a support case open since late last week, but I thought I'd ask the community if they can think of anything to check while I'm waiting... we're continuing to pull in corrupt (well, incomplete anyway) log data from these Windows forwarders so the delay in the back-and-forth-by-email isn't appealing.

Re: WinEventLog:* on Windows 2008 and Splunk 4.3.2 forwarders - Splunk could not get the description for this event.

rovechkin_splun — Tue, 08 May 2012 19:58:19 GMT

Can you enable splunk logging? In etc\log.cfg set the following flags to DEBUG
category.WinEventLogAgent=DEBUG

category.WinEventLogInputProcessor=DEBUG

category.WinEventLogChannel=DEBUG

category.WinEventLog=DEBUG

restart splunk and look into var\splunkd.log for possible errors

Re: WinEventLog:* on Windows 2008 and Splunk 4.3.2 forwarders - Splunk could not get the description for this event.

jeff — Wed, 09 May 2012 13:48:19 GMT

Yeah, I had done that for support... generates a lot of messages like the following:

DEBUG WinEventLogChannel - formatMessageByFlag: EvtFormatMessage returned no message, flag='(7)EvtFormatMessageProvider', channel='System', 'The handle is invalid.' WinEventLogChannel - getEventsNew: Failed to format source name of event log, channel='System', rec_id=434438 'The handle is invalid.'

several times for each event it couldn't process, but other than that not much useful. I downgraded to 4.3.1 on one forwarder yesterday and so far no errors...

Re: WinEventLog:* on Windows 2008 and Splunk 4.3.2 forwarders - Splunk could not get the description for this event.

marksnelling — Thu, 10 May 2012 09:30:33 GMT

I'm also getting a lot of these messages since upgrading my Windows Universal Forwarders to 4.3.2. I have a number of Universal Forwarders running on Windows 2008 R2 all forwarding to a single 4.3.2 indexer running on Linux.

Re: WinEventLog:* on Windows 2008 and Splunk 4.3.2 forwarders - Splunk could not get the description for this event.

the_wolverine — Fri, 11 May 2012 21:30:02 GMT

Our site has also encountered this. Smells like a bug as it has only occurred with the 4.3.2 UF, not with the previous version of UF (4.2.x)

Re: WinEventLog:* on Windows 2008 and Splunk 4.3.2 forwarders - Splunk could not get the description for this event.

the_wolverine — Fri, 11 May 2012 21:31:28 GMT

Have you considered downgrading the forwarder? We're not seeing this issue with UF 4.2.5.

Re: WinEventLog:* on Windows 2008 and Splunk 4.3.2 forwarders - Splunk could not get the description for this event.

jeff — Sun, 13 May 2012 01:26:38 GMT

yeah, support is finally filing a bug report...

Re: WinEventLog:* on Windows 2008 and Splunk 4.3.2 forwarders - Splunk could not get the description for this event.

jrodman — Mon, 14 May 2012 00:50:07 GMT

The likely relevant change was from 4.3.1 to 4.3.2; I would recommend using 4.3.1 for now. An example specific message which behaved that way would be useful.

Re: WinEventLog:* on Windows 2008 and Splunk 4.3.2 forwarders - Splunk could not get the description for this event.

jeff — Mon, 14 May 2012 12:52:52 GMT

You mean, other than the sample message in the post?... if you read through the rest of the post you'll see that I downgraded to 4.3.1 on key forwarders and the symptoms went away. Support is entering a bug report for me, as of late Friday afternoon.

Re: WinEventLog:* on Windows 2008 and Splunk 4.3.2 forwarders - Splunk could not get the description for this event.

jrodman — Mon, 14 May 2012 22:06:57 GMT

I meant the message as it should have been gathered; for example as it appears with 4.3.1 or in event log viewer. I'm working on the bug -- and more information will help.
; well the problem turned out to have not much relationship with the message, so nevermind.

Re: WinEventLog:* on Windows 2008 and Splunk 4.3.2 forwarders - Splunk could not get the description for this event.

jrodman — Wed, 16 May 2012 05:51:03 GMT

Hm, I answered this, but the comment system ate it whole. Trying again.

This is indeed a bug in 4.3.2 WinEventLog data acquisition code. It's now been identified and we can fix for the next release.

The code flow was altered in order to address performance concerns, which was a success. The performance for rapidly acquiring eventlog data should improve by around a factor of 2, which relieves stress on the wineventlog service (which was the bottleneck). The changes have to do with cacheing handles for data providers of windows eventlog strings.

Of course that's a poor consolation for correct operation. Unfortunately, the set of events we tested with did not have the distribution of data providers, so the problem wasn't identified internally.

For now please use 4.3.1 forwarders (or earlier) to acquire this data type.

Please note that this error message does not have a one to one correlation with this misbehavior. Other scenarios such as loading EVT files without the corresponding availble DLLs that provide the messages, or reading eventlogs for an application which has been subsequently uninstalled could (and do) produce the same message.

Re: WinEventLog:* on Windows 2008 and Splunk 4.3.2 forwarders - Splunk could not get the description for this event.

coreindustries — Sat, 19 May 2012 01:59:06 GMT

Where can I download 4.3.1?

Re: WinEventLog:* on Windows 2008 and Splunk 4.3.2 forwarders - Splunk could not get the description for this event.

farren — Mon, 21 May 2012 13:29:25 GMT

@coreindustries - http://www.splunk.com/page/previous_releases

Re: WinEventLog:* on Windows 2008 and Splunk 4.3.2 forwarders - Splunk could not get the description for this event.

gpt — Tue, 22 May 2012 15:38:07 GMT

Where can i get Universal forwarder 4.3.1 ??
Done!! Thank you!!! (Older versions link in the same download web ....)

Instead of the forwarder 4.3.1 the message of the event is not showed. Same message 'Splunk could not get ...'

Any idea?

Re: WinEventLog:* on Windows 2008 and Splunk 4.3.2 forwarders - Splunk could not get the description for this event.

bizza — Thu, 24 May 2012 10:56:15 GMT

I had the same issue, and now I'm downgrading the universal forwarder: hope it can solve the bug 🙂
Thank you

Re: WinEventLog:* on Windows 2008 and Splunk 4.3.2 forwarders - Splunk could not get the description for this event.

Lowell — Tue, 05 Jun 2012 13:36:14 GMT

Any timeline for the release of 4.3.3?

Re: WinEventLog:* on Windows 2008 and Splunk 4.3.2 forwarders - Splunk could not get the description for this event.

Lowell — Tue, 05 Jun 2012 13:43:53 GMT

For searching purposes, this issues is listed as SPL-51312.

Re: WinEventLog:* on Windows 2008 and Splunk 4.3.2 forwarders - Splunk could not get the description for this event.

jeff — Mon, 11 Jun 2012 14:13:37 GMT

In May I was told 4.3.3 was targeted for July.

Re: WinEventLog:* on Windows 2008 and Splunk 4.3.2 forwarders - Splunk could not get the description for this event.

mship — Tue, 19 Jun 2012 12:47:27 GMT

Rolling back to 4.3.1 UF did not work for me...any other suggestions?

Re: WinEventLog:* on Windows 2008 and Splunk 4.3.2 forwarders - Splunk could not get the description for this event.

JensT — Wed, 20 Jun 2012 16:08:00 GMT

Hello,

what the SPL number for this?

Kind Regards,

Jens