Sysmon events not being parsed — stuck as xmlwineventlog even with TA-microsoft-sysmon installed

[GuruDayal]

As i am struggling to solve this issue since 3 days. Need help in solving this ASAP, tried everything used GenAI tools. Still not resolved. Your Help will be much appreciated. 

Issue

Sysmon logs are ingesting fine but staying as raw XML with sourcetype xmlwineventlog.
Expected sourcetype (XmlWinEventLog:Microsoft-Windows-Sysmon/Operational) from the Splunk_TA_microsoft_sysmon is never applied — so no field extractions or CIM compliance.

Setup

* Indexer/HF: Ubuntu (Splunk Enterprise)

* Forwarder: Windows UF (universal forwarder, unparsed data)

* Inputs: WinEventLog://Microsoft-Windows-Sysmon/Operational

* Installed Add-ons:

Splunk_TA_microsoft_sysmon

Splunk_TA_windows

*Index: sysmon_logs

What’s happening

*In Events tab: raw XML visible (<Event xmlns=...>).

*In Statistics tab: only xmlwineventlog shows.

*Even after cleanup and restarts, Sysmon sourcetype never applied.

What I’ve tried

1.Verified ingestion: UF sends XML fine to indexer.

2.Checked TA inputs: correct source and renderXml=1.

3.Removed duplicate stanzas in system/local.

4.Confirmed TA configs load correctly via btool.

Created custom override (on indexer only):

# props.conf
[xmlwineventlog]
TRANSFORMS-fixsysmon = force_sysmon_sourcetype

# transforms.conf
[force_sysmon_sourcetype]
REGEX = (?i)Microsoft-Windows-Sysmon
SOURCE_KEY = _raw
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::XmlWinEventLog:Microsoft-Windows-Sysmon/Operational

*Verified via btool that stanza loads fine.

*Restarted Splunk — still no change.

*grep force_sysmon_sourcetype splunkd.log → no hits (transform not firing).

*stats count by sourcetype → only xmlwineventlog grows; Sysmon one frozen.

Need Help With-

*Why isn’t this index-time transform applying, even though it’s visible in btool and correctly configured?
*Is there a known precedence issue with xmlwineventlog that blocks sourcetype remapping?


index=sysmon_logs sourcetype=* | stats count by sourcetypeno events or statistics are shownExample event (raw XML view) Not parsingprops.conf + transforms.conf contents

[PickleRick]

1. XmlWinEventLog (or whatever the case is - it's not important) is the right sourcetype for XML formatted windows events. It's the source that distinguishes from which eventlog channel the event came.

2. Your data seems to be parsed properly (you have your fields extracted). It is not "prettyfied" in the UI but that's normal behaviour.

3. "Need this ASAP" doesn't work here. It's not a free support service. You want something done quickly? Find and engage your local Splunk Partner.

[GuruDayal]

Need Solution as earliest as possible, Help will be much appreciated as i am struggling since days of this issue, cannot go to the next step of my project.

Here’s the latest update on my Sysmon parsing issue:

I’ve now confirmed the following:

• Sysmon TA v5.0.0 (Splunk_TA_microsoft_sysmon) is installed on my Splunk VM (which acts as both Indexer + Search Head).

• UF on Windows forwards Sysmon logs using:

  [WinEventLog://Microsoft-Windows-Sysmon/Operational]
  disabled = false renderXml = 1
  sourcetype = XmlWinEventLog
  source = XmlWinEventLog:Microsoft-Windows-Sysmon/Operational
  index = sysmon_logs
  
  

• Data ingestion works — events arrive fine under
  sourcetype=XmlWinEventLog and
  source=XmlWinEventLog:Microsoft-Windows-Sysmon/Operational.

• TA overrides removed (TA-sysmon-overrides.disabled) — verified no local props/transforms overriding the Sysmon TA.

• CIM installed — | tstats count from datamodel=Endpoint.Processes shows results.

• btool verification shows correct sourcetype chain; no misconfig or duplicates.

• However, Sysmon events are still shown as raw XML, and none of the field extractions (like Image, ParentImage, CommandLine) appear — even though ingestion and sourcetype are correct.

So far, I’ve verified:
✅ ingestion and indexing
✅ sourcetype normalization
✅ TA version and configuration
✅ CIM presence and tstats validation
✅ no conflicting props/transforms

Yet parsing doesn’t occur at search-time.
Could this be an issue with the Sysmon TA’s extraction logic not triggering for XmlWinEventLog events (even though the sourcetype matches)?
Any next steps or validation checks you recommend to debug why XML extraction isn’t being applied?

Attached screenshots are merged in 1 png:

1. Raw Sysmon XML event from index=sysmon_logs

2. Search showing sourcetype/source fields

3. btool props list XmlWinEventLog --debug output

[PrewinThomas]

@GuruDayal 

Did you verify your logs for cim complaint?
Eg: 

| tstats count from datamodel=Endpoint.Processes where index=sysmon_logs by Processes.process_name

Also why you are forcing sourcetype XmlWinEventLog:Microsoft-Windows-Sysmon/Operational ?
Newer add-on sourcetype is XmlWinEventLog with source="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational"
#https://splunk.github.io/splunk-add-on-for-microsoft-sysmon/Sourcetypes/

No need to force your sourcetype for the sysmon. Your UF inputs.conf should be something like this below,

[WinEventLog://Microsoft-Windows-Sysmon/Operational]
disabled = false
renderXml = 1
source = XmlWinEventLog:Microsoft-Windows-Sysmon/Operational
index=sysmon_logs

And put sysmon add-on #https://splunkbase.splunk.com/app/5709 in your HF/Indexer where your parsing is happening

Regards,
Prewin
If this answer helped you, please consider marking it as the solution or giving a Karma. Thanks!

[GuruDayal]

Thanks a lot for the clarification earlier — that really helped fix most of the issues.

Here’s what I’ve verified and observed so far:

• I’m using Splunk_TA_microsoft_sysmon v5.0.0 (no overrides anymore).

• Events are now correctly coming in with:

  • sourcetype=XmlWinEventLog

  • source=XmlWinEventLog:Microsoft-Windows-Sysmon/Operational

• The Splunk Common Information Model (SA-CIM) add-on is now installed.

• The CIM check works fine —

  | tstats count from datamodel=Endpoint.Processes where index=sysmon_logs by Processes.process_name

  shows valid results when using a historical time range (error earlier was found due to real-time search).

However, I’m still seeing the following issues:

• In the Events tab (index=sysmon_logs), logs are displayed as raw XML (not parsed).

• The Field values shows two sourcetypes for the same index:

  • XmlWinEventLog

  • xmlwineventlog

• When I use index=sysmon_logs | head 5, I only see XmlWinEventLog, but not the parsed fields I’d expect from the Sysmon TA.

So it seems parsing and field extraction are only partially applied — possibly due to inconsistent sourcetype assignment (xmlwineventlog vs XmlWinEventLog).

Could you please advise what could be causing both sourcetypes to appear and why the XmlWinEventLog events still show raw XML in the Events tab, even though CIM and the TA are installed correctly?

I’d really appreciate any additional guidance — your earlier input already helped a lot in getting Sysmon ingestion and CIM mapping working.

[PrewinThomas]

@GuruDayal 
Did you install Splunk_TA_windows add-on also? XmlWinEventLog sourcetype require this add-on also to parse everything. 

With Splunk_TA_windows and Splunk_TA_microsoft_sysmon add-on it should parse as expected.

Regards,
Prewin
If this answer helped you, please consider marking it as the solution or giving a Karma. Thanks!