I’ve inherited a fleet of about 150 Windows Servers, all configured identically — same Deployment Server, TAs, inputs.conf/outputs.conf, etc. Out of the 150, around 10-12 systems are sending most Windows logs as expected, except for Security logs (WinEventLog:Security).
I’ve already tried the basics like rebooting and reinstalling the forwarder, but no go. I’m leaning toward a possible permissions issue but not sure where to start troubleshooting from here.
Hi @samalchow
Please let us know if you are getting _internal logs for the UFs not sending the windows data as this might help determine if its a Windows permissions issue on the collection of event data, or an issue sending 🙂
Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards
Will
On some our Windows UF hosts, we were getting System events but no Security events. Our Windows admin noticed that the Splunk service account was running as an NT service. After changing the service account to LocalSystem, the Windows UF hosts started sending their security events.
Here is more about selecting correct user in windows environment https://help.splunk.com/en/splunk-enterprise/get-started/install-and-upgrade/9.4/install-splunk-ente...
The problem is likely that the program (Splunk UF) doesn't have the right access. Windows keeps very strict records of security events, and needs to be allowed to see them.
As windows user account either local account or domain account with:
Required Local/Domain Security Policy user rights assignments for the splunkd or splunkforwarder services
Additional activity for data required in use-cases
Audit process tracking | Microsoft Learn
If other logs from that forwarder (especially other winevent logs) are properly forwarded, it's most probably a permissions issue. The user UF is running as must be able to read Security log (as typically access to this log is limited whereas to System or Application is much wider open). The rights to eventlog channels are assigned by crafting proper acl in a specific registry key (sorry, don't remember which one) but if your ACLs haven't been tampered with, you can simply add the UF's user to the Event Log Readers local group.
Hi @samalchow
Are you getting logs into _internal for the other hosts? This might help determine if the issue is with the inputs or the outputs of the UFs.
Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards
Will