This Crashed Our Server Last Night, Completely, Is There Any Particular Reason Why This Happened?

[jlvix1]

Faulting application name: splunk-winevtlog.exe, version: 1541.256.22575.14967, time stamp: 0x582f3e24
Faulting module name: KERNELBASE.dll, version: 6.3.9600.18340, time stamp: 0x57366075
Exception code: 0xeeab5254
Fault offset: 0x0000000000008a5c
Faulting process id: 0x774
Faulting application start time: 0x01d24fc49cf57f77
Faulting application path: C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe
Faulting module path: C:\Windows\system32\KERNELBASE.dll
Report Id: 002d6d22-f244-11e6-8145-065cadcce291
Faulting package full name:
Faulting package-relative application ID:

The exception code resolves to "Stack Overflow". I have since stopped windows event log collection on all systems, as I understand this is a windows event log collector component issue.

The light forwarder version we are using is 6.5.1

[jtacy]

By crash do you mean a BSOD or something else? I've never seen anything like this and we run thousands of UFs on Windows so I wonder if the event log entry is illustrating a symptom rather than the cause of the crash. I would recommend engaging Splunk Support about this. If you don't know the exact timestamp of the crash, it may be interesting to see what events Splunk indexed just before the crash. This search may help:

yoursearch | rename _indextime AS indextime | convert ctime(indextime)

This will create a field called indextime that will give you the time that Splunk indexed the event rather than the time of the event itself. Good luck!

[jlvix1]

Hi, no this was a crash of the UF implicating KERNELBASE.dll - the stack overflow caused other issues on the server, clearly there was a memory leak of some sort that affected everything else on the server.

I will have a look at that, thanks.