Sysmon events not getting indexed

[corti77]

Hi,

I am deploying sysmon all acrros our company but for some reason the sysmon events are not getting indexed

Our deployment is the following:

• Splunk 9.0.5 running on Windows server
  • sysmon index created manually in Splunk.
  • inbound firewall rules created allowing traffic TCP in port 9997
• Sysmon TA installed in the server in C:\Program Files\Splunk\etc\deployment-apps\Splunk_TA_microsoft_sysmon
  • default/input.cont enabled (by default)

[WinEventLog://Microsoft-Windows-Sysmon/Operational]
disabled = false
renderXml = 1
source = XmlWinEventLog:Microsoft-Windows-Sysmon/Operational
​

• local/input.conf containing 

[WinEventLog://Microsoft-Windows-Sysmon/Operational]
index=sysmon​

• Splunk Universal forwarder 9.1.0 deployed in all hosts
  • All UF are reporting correctly to Splunk
  • confirmed that sysmon TA is present in all hosts , deployed via forwarder management using a server class
  • /etc/system/default/ouputs.conf is pointing to the right splunk server

[tcpout]
defaultGroup = default-autolb-group

[tcpout:default-autolb-group]
server = xxxxxx:9997

[tcpout-server://xxxxxx:9997]​

• Sysmon 15 deployed in all hosts
  
  • confirmed that the events are being created locally in the hosts in Microsoft-->Windows-->Sysmon tree

But no single event appears in the sysmon index 😞

Does anyone have any idea or suggestion of what might be missing?

many thanks

[DanielAmlung]

So we did a little more digging. The magic in this case is called "channelAccess". This setting basically grants access to specific event logs for "normal user" accounts. High value event logs, for example: security event log, are only available for higher privileged accounts. The sysmon event log is considered "high value". So when you check the "channelAccess" settings you get this by default:

But Microsoft allows you to modify this setting. So we modified the "channelAccess" settings based on the "powershell operational" settings:

After you set the setting, restart the SplunkForwarder Service. And voila - you have access to the sysmon event log without adding the virtual account to a specific group.

[roshan]

This worked for us. We changed the ChannelAccess of Sysmon to that of Powershell.
Run the following on cmd,

wevtutil sl "Microsoft-Windows-Sysmon/Operational" /ca:<Powershell ChannelAccess>

[MaverickT]

If you want to run SplunkForwarder with virtual account (which is recommended if you want to follow princpile of the least privileges) there is also a way to enable reading of sysmon logs. NT SERVICE/SplunkFowarder needs to be added to Event Log Readers group.

One of the ways is to add it to the Group policy and deploy it accross your environment where your Forwarders are installed.

 

[dzampino]

This was the fix we were looking for. I ended up using group policy preferences to add NT SERVICE\SplunkForwarer to the Event Log Readers group instead of using Restricted Groups (defining members in Restricted Groups will remove members already in the group not listed, so be cautious).

[arsidiq]

I have the same issue, how do you all solve it guys?

[corti77]

in my case the issue was the user running the splunk universal forwarder service.

open  the services manager and check that, it should be SYSTEM or any user with local admin rights

[corti77 · Accepted solution]

I finally found the reason. it was due to the user configured to run the Splunk forwarder windows service. It was a local user account without necessary rights. I changed it to local system account and the events started to flow in.

[Zer0sss]

hi @corti77 

I'm getting the same error, but my Splunk UF is running as Administrator

But I still get the same error. I wonder if there is any other way to fix this error ?

Thanks

[corti77]

try to configure it with the user SYSTEM. if the issue persists, check the local logs of the universal forwarder located in c:\program files\splunk\var\log\splunkd.log

[corti77]

I found the reason but not the solution. at the host level , the splunk forwarder does not have access to the sysmon event logs for some unknown reason.

any idea?

08-25-2023 16:34:02.254 +0200 ERROR ExecProcessor [1340 ExecProcessor] - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" splunk-winevtlog - WinEventLogChannel::subscribeToEvtChannel: Could not subscribe to Windows Event Log channel 'Microsoft-Windows-Sysmon/Operational'
08-25-2023 16:34:02.254 +0200 ERROR ExecProcessor [1340 ExecProcessor] - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" splunk-winevtlog - WinEventLogChannel::init: Init failed, unable to subscribe to Windows Event Log channel 'Microsoft-Windows-Sysmon/Operational': errorCode=5

[m_zandinia]

Do you find any solution for this?

I have some UFs that run with local system and they can send sysmon logs but I have some UFs that run with virtual account and therefore they can't send sysmon logs.

I have the same message as you.

[DanielAmlung]

Hi,

so I also had the same problem. I tested several setups and what worked was the solution provided by MaverickT. Just create a GPO and add the virtual Account to the "Event Log Readers" Group. This does the trick.

It seems that the privilege "SeSecurityPrivilege" isnt enough to read the sysmon event log.

 

Which is weird, because all the other logs are readable. I can read power shell logs with this settings, but not the sysmon logs.