I'm having som issues with the application log on some of our windows servers getting spammed with the following messages:
Faulting application name: splunk-winevtlog.exe, version: 1794.768.23581.39240, time stamp: 0x5c1d9d74 Faulting module name: KERNELBASE.dll, version: 6.3.9600.19724, time stamp: 0x5ec5262a Exception code: 0xeeab5254 Fault offset: 0x0000000000007afc Faulting process id: 0x3258 Faulting application start time: 0x01d787a1d9f141cd Faulting application path: C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe Faulting module path: C:\Windows\system32\KERNELBASE.dll Report Id: 18687572-f395-11eb-8131-005056b32672 Faulting package full name: Faulting package-relative application ID:
Always followed by a 1001 information event like so:
Fault bucket , type 0 Event Name: APPCRASH Response: Not available Cab Id: 0 Problem signature: P1: splunk-winevtlog.exe P2: 1794.768.23581.39240 P3: 5c1d9d74 P4: KERNELBASE.dll P5: 6.3.9600.19724 P6: 5ec5262a P7: eeab5254 P8: 0000000000007afc P9: P10: Attached files: These files may be available here: C:\ProgramData\Microsoft\Windows\WER\ReportQueue\AppCrash_splunk-winevtlog_32b957db7bcb27fbdcdd5be64aea86e1b639666_0170a0ed_a993dd7e Analysis symbol: Rechecking for solution: 0 Report Id: 18687572-f395-11eb-8131-005056b32672 Report Status: 4100 Hashed bucket:
I've tried a lot of changes to the Universal Forwarder configuration but nothing i do removes these message. The only thing i've noticed that can helt to remove these messages is by lowering the memory consumption on the server. So far the servers i've seen with these message in the application log are running at 70% and more memory consumption. But 70% memory consumption seems to be normal and i don't see why this should cause the splunk-winevtlog.exe to crash (as often as every minute).
Our version of Splunk Universal Forwarder is 7.2.3. I've checked the "known issues" on splunk docs but can't fint anything related to memory issues for this version.
I'm thinking about upgrading the Universal Forwarder to a newer version, but that's just because i can't think og anything else to try. Do anyone else experience this and know what can be done?
As a side note: Splunk internal shows absolutely nothing. There are no warnings or errors at all in the internal log on these servers. But the event spamming (crashes) are still logged in the windows application log. Splunk itself does not log or detect a crash it seems?
@mykol_j - It could be very likely permission issue on Windows. Try with any other test instance, run Splunk with previledged user.
If nothing works, you can create Support ticket.
I hope this helps!!!
Good point. I'm afraid if that works, it's still not a fix as we can't run processes with elevated privs... I'll still see about testing it though.
wow, almost two years and now answer?
I'm getting this too, on about 383 different UF clients (out of about a1,000).
I'm using the latest 18.104.22.168...