Deployment Architecture

This Crashed Our Server Last Night, Completely, Is There Any Particular Reason Why This Happened?

jlvix1
Communicator

Faulting application name: splunk-winevtlog.exe, version: 1541.256.22575.14967, time stamp: 0x582f3e24
Faulting module name: KERNELBASE.dll, version: 6.3.9600.18340, time stamp: 0x57366075
Exception code: 0xeeab5254
Fault offset: 0x0000000000008a5c
Faulting process id: 0x774
Faulting application start time: 0x01d24fc49cf57f77
Faulting application path: C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe
Faulting module path: C:\Windows\system32\KERNELBASE.dll
Report Id: 002d6d22-f244-11e6-8145-065cadcce291
Faulting package full name:
Faulting package-relative application ID:

The exception code resolves to "Stack Overflow". I have since stopped windows event log collection on all systems, as I understand this is a windows event log collector component issue.

The light forwarder version we are using is 6.5.1

Tags (1)
0 Karma

jtacy
Builder

By crash do you mean a BSOD or something else? I've never seen anything like this and we run thousands of UFs on Windows so I wonder if the event log entry is illustrating a symptom rather than the cause of the crash. I would recommend engaging Splunk Support about this. If you don't know the exact timestamp of the crash, it may be interesting to see what events Splunk indexed just before the crash. This search may help:

yoursearch | rename _indextime AS indextime | convert ctime(indextime)

This will create a field called indextime that will give you the time that Splunk indexed the event rather than the time of the event itself. Good luck!

jlvix1
Communicator

Hi, no this was a crash of the UF implicating KERNELBASE.dll - the stack overflow caused other issues on the server, clearly there was a memory leak of some sort that affected everything else on the server.

I will have a look at that, thanks.

0 Karma
Get Updates on the Splunk Community!

Splunk Decoded: Service Maps vs Service Analyzer Tree View vs Flow Maps

It’s Monday morning, and your phone is buzzing with alert escalations – your customer-facing portal is running ...

What’s New in Splunk Observability – September 2025

What's NewWe are excited to announce the latest enhancements to Splunk Observability, designed to help ITOps ...

Fun with Regular Expression - multiples of nine

Fun with Regular Expression - multiples of nineThis challenge was first posted on Slack #regex channel ...