Getting Data In

Some Windows UF not sending Security logs

samalchow
Observer

I’ve inherited a fleet of about 150 Windows Servers, all configured identically — same Deployment Server, TAs, inputs.conf/outputs.conf, etc. Out of the 150, around 10-12 systems are sending most Windows logs as expected, except for Security logs (WinEventLog:Security).

I’ve already tried the basics like rebooting and reinstalling the forwarder, but no go. I’m leaning toward a possible permissions issue but not sure where to start troubleshooting from here.

Labels (2)
0 Karma

livehybrid
SplunkTrust
SplunkTrust

Hi @samalchow 

Please let us know if you are getting _internal logs for the UFs not sending the windows data as this might help determine if its a Windows permissions issue on the collection of event data, or an issue sending 🙂

Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards

Will

0 Karma

computermathguy
Path Finder

On some our Windows UF hosts, we were getting System events but no Security events.  Our Windows admin noticed that the Splunk service account was running as an NT service.  After changing the service account to LocalSystem, the Windows UF hosts started sending their security events.

0 Karma

isoutamo
SplunkTrust
SplunkTrust
0 Karma

kiran_panchavat
Champion

@samalchow 

The problem is likely that the program (Splunk UF) doesn't have the right access. Windows keeps very strict records of security events, and needs to be allowed to see them.

As windows user account either local account or domain account with:

  1. Full control over the Splunk installation directory.
  2. Read access to any files that you want to index. e.g. Windows application, security, system, application logs, MSSQL logs and so on.
  3. For standard naming convention it's recommended to create user "Splunk" with the required privileges.

Required Local/Domain Security Policy user rights assignments for the splunkd or splunkforwarder services

  1. Permission to log on as a service.
  2. Permission to log on as a batch job
  3. Permission to replace a process-level token.
  4. Permission to act as part of the operating system.
  5. Permission to bypass traverse checking

Additional activity for data required in use-cases

  • Enable Windows event and audit logs
  • Turn on Process Tracking in your Windows Audit logs (Event ID 4688)

Audit process tracking | Microsoft Learn

  • Windows Security Event ID 4648 tracks the explicit use of credentials, as in a run as event or batch login from a scheduled task. You can enable this from your Windows Logon Event policy configuration

 

Did this help? If yes, please consider giving kudos, marking it as the solution, or commenting for clarification — your feedback keeps the community going!
0 Karma

PickleRick
SplunkTrust
SplunkTrust

If other logs from that forwarder (especially other winevent logs) are properly forwarded, it's most probably a permissions issue. The user UF is running as must be able to read Security log (as typically access to this log is limited whereas to System or Application is much wider open). The rights to eventlog channels are assigned by crafting proper acl in a specific registry key (sorry, don't remember which one) but if your ACLs haven't been tampered with, you can simply add the UF's user to the Event Log Readers local group.

0 Karma

livehybrid
SplunkTrust
SplunkTrust

Hi @samalchow 

Are you getting logs into _internal for the other hosts? This might help determine if the issue is with the inputs or the outputs of the UFs.

Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards

Will

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

Observe and Secure All Apps with Splunk

 Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

What's New in Splunk Observability - August 2025

What's New We are excited to announce the latest enhancements to Splunk Observability Cloud as well as what is ...

Introduction to Splunk AI

How are you using AI in Splunk? Whether you see AI as a threat or opportunity, AI is here to stay. Lucky for ...