Getting Data In

Sysmon events not getting indexed

corti77
Communicator

Hi,

I am deploying sysmon all acrros our company but for some reason the sysmon events are not getting indexed

Our deployment is the following:

  • Splunk 9.0.5 running on Windows server
    • sysmon index created manually in Splunk.
    • inbound firewall rules created allowing traffic TCP in port 9997
  • Sysmon TA installed in the server in C:\Program Files\Splunk\etc\deployment-apps\Splunk_TA_microsoft_sysmon
    • default/input.cont enabled (by default)

 

[WinEventLog://Microsoft-Windows-Sysmon/Operational]
disabled = false
renderXml = 1
source = XmlWinEventLog:Microsoft-Windows-Sysmon/Operational
​

 

  • local/input.conf containing 

 

[WinEventLog://Microsoft-Windows-Sysmon/Operational]
index=sysmon​

 

  • Splunk Universal forwarder 9.1.0 deployed in all hosts
    • All UF are reporting correctly to Splunk
    • confirmed that sysmon TA is present in all hosts , deployed via forwarder management using a server class
    • /etc/system/default/ouputs.conf is pointing to the right splunk server

 

[tcpout]
defaultGroup = default-autolb-group

[tcpout:default-autolb-group]
server = xxxxxx:9997

[tcpout-server://xxxxxx:9997]​

 

  • Sysmon 15 deployed in all hosts
    • confirmed that the events are being created locally in the hosts in Microsoft-->Windows-->Sysmon tree

But no single event appears in the sysmon index 😞

Does anyone have any idea or suggestion of what might be missing?

many thanks

Labels (1)
0 Karma
1 Solution

corti77
Communicator

I finally found the reason. it was due to the user configured to run the Splunk forwarder windows service. It was a local user account without necessary rights. I changed it to local system account and the events started to flow in.

View solution in original post

DanielAmlung
Explorer

So we did a little more digging. The magic in this case is called "channelAccess". This setting basically grants access to specific event logs for "normal user" accounts. High value event logs, for example: security event log, are only available for higher privileged accounts. The sysmon event log is considered "high value". So when you check the "channelAccess" settings you get this by default:

DanielAmlung_0-1716820326229.png

But Microsoft allows you to modify this setting. So we modified the "channelAccess" settings based on the "powershell operational" settings:

DanielAmlung_1-1716820541465.png

DanielAmlung_2-1716820576102.png

After you set the setting, restart the SplunkForwarder Service. And voila - you have access to the sysmon event log without adding the virtual account to a specific group.

 

0 Karma

MaverickT
Communicator

If you want to run SplunkForwarder with virtual account (which is recommended if you want to follow princpile of the least privileges) there is also a way to enable reading of sysmon logs. NT SERVICE/SplunkFowarder needs to be added to Event Log Readers group.

One of the ways is to add it to the Group policy and deploy it accross your environment where your Forwarders are installed.

image.png

arsidiq
New Member

I have the same issue, how do you all solve it guys?

0 Karma

corti77
Communicator

in my case the issue was the user running the splunk universal forwarder service.

open  the services manager and check that, it should be SYSTEM or any user with local admin rights

Tags (1)
0 Karma

corti77
Communicator

I finally found the reason. it was due to the user configured to run the Splunk forwarder windows service. It was a local user account without necessary rights. I changed it to local system account and the events started to flow in.

Zer0sss
Loves-to-Learn Lots

hi @corti77 

I'm getting the same error, but my Splunk UF is running as Administrator

But I still get the same error. I wonder if there is any other way to fix this error ?

Thanks

0 Karma

corti77
Communicator

try to configure it with the user SYSTEM. if the issue persists, check the local logs of the universal forwarder located in c:\program files\splunk\var\log\splunkd.log

0 Karma

corti77
Communicator

I found the reason but not the solution. at the host level , the splunk forwarder does not have access to the sysmon event logs for some unknown reason.

 

any idea?

 

08-25-2023 16:34:02.254 +0200 ERROR ExecProcessor [1340 ExecProcessor] - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" splunk-winevtlog - WinEventLogChannel::subscribeToEvtChannel: Could not subscribe to Windows Event Log channel 'Microsoft-Windows-Sysmon/Operational'
08-25-2023 16:34:02.254 +0200 ERROR ExecProcessor [1340 ExecProcessor] - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe"" splunk-winevtlog - WinEventLogChannel::init: Init failed, unable to subscribe to Windows Event Log channel 'Microsoft-Windows-Sysmon/Operational': errorCode=5
0 Karma

m_zandinia
Path Finder

Do you find any solution for this?

I have some UFs that run with local system and they can send sysmon logs but I have some UFs that run with virtual account and therefore they can't send sysmon logs.

I have the same message as you.

0 Karma

DanielAmlung
Explorer

Hi,

so I also had the same problem. I tested several setups and what worked was the solution provided by MaverickT. Just create a GPO and add the virtual Account to the "Event Log Readers" Group. This does the trick.

It seems that the privilege "SeSecurityPrivilege" isnt enough to read the sysmon event log.

pro_ex_sysm.png

pro_ex_rights.png

Which is weird, because all the other logs are readable. I can read power shell logs with this settings, but not the sysmon logs.

Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...