topic Universal forwarder version 10 memory leak in Splunk Enterprise

Universal forwarder version 10 memory leak

Ixionz — Thu, 21 Aug 2025 16:19:48 GMT

I am currently in the testing phase of getting our universal forwarders to a more standardized version (either 9.4.4 or version 10), however when I roll out the new version to any VM's splunkforwarder chews up over 80% of memory which causes overall memory utilization to be around 100% constantly which I am forced to rollback to version 9.4.4

Nothing has been changed at all except the version.

Is anyone else experiencing similar behavior when they upgrade to version 10 or even do a new install, or has anyone else seen this behavior out there (not necessarily VM's but maybe physical boxes) as i don't want to roll something out to our environment and causes more problems than solutions.

Re: Universal forwarder version 10 memory leak

livehybrid — Fri, 22 Aug 2025 08:56:22 GMT

Hi @Ixionz

Are you able to confirm please the name of the process(es) running which consume this amount of memory? And also the total amount of memory on these VMs?

🌟 Did this answer help you? If so, please consider:

Adding karma to show it was useful
Marking it as the solution if it resolved your issue
Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

Re: Universal forwarder version 10 memory leak

isoutamo — Fri, 22 Aug 2025 12:14:53 GMT

Please create a support ticket.

Anyhow it's best practice to wait something like X.0.3 or even X.1.2 or similar before go into production. There have been almost every time when a new version has launched more or less nasty and critical bugs.

Re: Universal forwarder version 10 memory leak

darren — Mon, 25 Aug 2025 13:43:18 GMT

I was told to try the fix in:

https://community.splunk.com/t5/Splunk-Enterprise/URGENT-All-splunk-forwarders-upgraded-to-10-0-version-are/td-p/751274

"Disabled the

evt_resolve_ad_obj = 0

in Splunk_TA_windows app , logs have now ceased. "

For reference, this is the ticket I made. Luckily, we were able to catch this issue in dev before deploying 10.0.0.0 to prod.

https://community.splunk.com/t5/Splunk-Enterprise/In-UniversalForwarder-10-0-0-0-splunk-winevtlog-exe-process/td-p/752063

Re: Universal forwarder version 10 memory leak

darren — Mon, 25 Aug 2025 13:55:21 GMT

However, if we do the below "fix", then AD SID and AD GUID strings won't be resolved to the actual AD names, which would be really annoying. I think we're going to hold off on 10.0.0.0 until the evt_resolve_ad_obj feature is fixed and working again without crashing our servers.

[WinEventLog://Security] stanzas inside of inputs.conf:

evt_resolve_ad_obj = 0

Re: Universal forwarder version 10 memory leak

darren — Mon, 29 Sep 2025 21:14:43 GMT

I see this article: https://splunk.my.site.com/customer/s/article/High-CPU-and-Memory-Usage-After-Splunk-UF-10-Upgrade

I've just tested out 9.4.5.0, and having same issue with it crashing servers.

9.4.4.0 seems to be safe for us.

So far we've seen these crashes on Windows 2016, not sure if it affects other OS versions or not.

Re: Universal forwarder version 10 memory leak

Dave737 — Thu, 06 Nov 2025 16:09:18 GMT

I upgraded to Splunk Forwarder 10.0.1 yesterday on a PC running Windows10 with 32GB of RAM. The process name is "Monitor windows event log" which is called from "splunk-wineventlog.exe" This process sat consuming over 28GB of RAM!

I reverted back to 9.4.3 which consumes about 150MB of RAM.

This seems to affect physical servers, VM's and PC's. Luckily I didn't deploy it to too many machines and it's strange that some are running the update with no memory issues as yet. I have had to revert the forwarder on 3 machines but still testing on half a dozen others.