topic Re: Getting time from XML Property in Getting Data In

Getting time from XML Property

Ulwur2 — Wed, 02 Oct 2024 09:06:15 GMT

I'm experimenting with doing ETW logging of Microsoft IIS, where the IIS log ends up as XML in a windows eventlog.

But I have problems getting Splunk to use the correct timestamp field, Splunk uses the TimeCreated property for eventtime (_time), and not the date and time properties that indicate when IIS served the actual webpage.

An example:
<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-IIS-Logging' Guid='{7e8ad27f-b271-4ea2-a783-a47bde29143b}'/><EventID>6200</EventID><Version>0</Version><Level>4</Level><Task>0</Task><Opcode>0</Opcode><Keywords>0x8000000000000000</Keywords><TimeCreated SystemTime='2024-04-04T12:23:43.811459900Z'/><EventRecordID>11148</EventRecordID><Correlation/><Execution ProcessID='1892' ThreadID='3044'/><Channel>Microsoft-IIS-Logging/Logs</Channel><Computer>sw2iisxft</Computer><Security UserID='S-1-5-18'/></System><EventData><Data Name='EnabledFieldsFlags'>2149961727</Data><Data Name='date'>2024-04-04</Data><Data Name='time'>12:23:37</Data><Data Name='cs-username'>ER\4dy</Data><Data Name='s-sitename'>W3SVC5</Data><Data Name='s-computername'>sw2if</Data><Data Name='s-ip'>192.168.32.86</Data><Data Name='cs-method'>GET</Data><Data Name='cs-uri-stem'>/</Data><Data Name='cs-uri-query'>blockid=2&roleid=8&logid=21</Data><Data Name='sc-status'>200</Data><Data Name='sc-win32-status'>0</Data><Data Name='sc-bytes'>39600</Data><Data Name='cs-bytes'>984</Data><Data Name='time-taken'>37</Data><Data Name='s-port'>443</Data><Data Name='csUser-Agent'>Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/123.0.0.0+Safari/537.36+Edg/123.0.0.0</Data><Data Name='csCookie'>-</Data><Data Name='csReferer'>https://tsidologg/?blockid=2&roleid=8</Data><Data Name='cs-version'>-</Data><Data Name='cs-host'>-</Data><Data Name='sc-substatus'>0</Data><Data Name='CustomFields'>X-Forwarded-For - Content-Type - https on host tsidologg</Data></EventData></Event>

I've tried every combination in props.conf that I can think of

This should work, but doesen't..

TIME_PREFIX = <Data Name='date'> MAX_TIMESTAMP_LOOKAHEAD = 100 TIME_FORMAT =<Data Name='date'>%Y-%m-%d</Data><Data Name='time'>%H:%M:%S</Data> TZ = UTC

Any ideas?

Re: Getting time from XML Property

richgalloway — Thu, 04 Apr 2024 14:22:21 GMT

There should be no overlap between the TIME_PREFIX and TIME_FORMAT settings. Splunk skips past TIME_PREFIX and then starts looking for text that matches TIME_FORMAT. Since "<Data Name='date'>" doesn't appear twice in a row, there is no match for the timestamp. Try these settings:

TIME_PREFIX = <Data Name='date'> MAX_TIMESTAMP_LOOKAHEAD = 100 TIME_FORMAT = %Y-%m-%d</Data><Data Name='time'>%H:%M:%S TZ = UTC

Re: Getting time from XML Property

Ulwur2 — Fri, 05 Apr 2024 09:41:57 GMT

Thanks!

I have tried your setting, and unfortenatly it still doesn't work.

I have also discovered that xml data from sysmon have the same exact problem, won't pick up the time from the expected field from the xml data.

sysmon looks like this, and it matches the text in the xml.

[source::XmlWinEventLog:Microsoft-Windows-Sysmon/Operational]
TIME_PREFIX = <Data Name='UtcTime'>
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N
TZ = UTC

I have used btool to look for any other stanzas that would cause this, for example for the common xmlwineventlog sourcetype but haven't found anyting.

Tips for debugging this welcome!