AD FS Audit Logs (ADFS Audit) XML issues: Why is the RAW not parsing out the XML?

[samsonusmc]

I am using Splunk TA for Windows infrastructure configured to consume the XML logs.
The problem is the RAW doesn't parse out the XML that is contained w/in the XML log very well (see raw output below)

It parses the "Outer" XML fine, but the "Inner" XML, not so much. (See all the lt; and gt;) > and < is just to get through this WYSIWYG editor

Anyone have good advice on how to get splunk to parse and store this properly?

<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'>
    <System>
        <Provider Name='AD FS Auditing'/>
        <EventID Qualifiers='0'>1200</EventID>
        <Level>0</Level>
        <Task>3</Task>
        <Keywords>0x80a0000000000000</Keywords>
        <TimeCreated SystemTime='2020-05-31T20:31:28.875321100Z'/>
        <EventRecordID>27322577</EventRecordID>
        <Channel>Security</Channel>
        <Computer>Computer</Computer>
        <Security UserID='S-1-5-21----SID'/>
    </System>
    <EventData>
        <Data>3fb4c7cb-865b-4d89-3e02-0080010000b3</Data>
        <Data>&amp;lt;?xml version="1.0" encoding="utf-16"?&amp;gt;
&amp;lt;AuditBase xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="AppTokenAudit"&amp;gt;
  &amp;lt;AuditType&amp;gt;AppToken&amp;lt;/AuditType&amp;gt;
  &amp;lt;AuditResult&amp;gt;Success&amp;lt;/AuditResult&amp;gt;
  &amp;lt;FailureType&amp;gt;None&amp;lt;/FailureType&amp;gt;
  &amp;lt;ErrorCode&amp;gt;N/A&amp;lt;/ErrorCode&amp;gt;
  &amp;lt;ContextComponents&amp;gt;
    &amp;lt;Component xsi:type="ResourceAuditComponent"&amp;gt;
      &amp;lt;RelyingParty&amp;gt;RelyingParty&amp;lt;/RelyingParty&amp;gt;
      &amp;lt;ClaimsProvider&amp;gt;AD AUTHORITY&amp;lt;/ClaimsProvider&amp;gt;
      &amp;lt;UserId&amp;gt;UserId&amp;lt;/UserId&amp;gt;
    &amp;lt;/Component&amp;gt;
    &amp;lt;Component xsi:type="AuthNAuditComponent"&amp;gt;
      &amp;lt;PrimaryAuth&amp;gt;http://schemas.microsoft.com/ws/2008/06/identity/authenticationmethod/windows&amp;lt;/PrimaryAuth&amp;gt;
      &amp;lt;DeviceAuth&amp;gt;false&amp;lt;/DeviceAuth&amp;gt;
      &amp;lt;DeviceId&amp;gt;N/A&amp;lt;/DeviceId&amp;gt;
      &amp;lt;MfaPerformed&amp;gt;false&amp;lt;/MfaPerformed&amp;gt;
      &amp;lt;MfaMethod&amp;gt;N/A&amp;lt;/MfaMethod&amp;gt;
      &amp;lt;TokenBindingProvidedId&amp;gt;true&amp;lt;/TokenBindingProvidedId&amp;gt;
      &amp;lt;TokenBindingReferredId&amp;gt;false&amp;lt;/TokenBindingReferredId&amp;gt;
      &amp;lt;SsoBindingValidationLevel&amp;gt;TokenBoundAndValid&amp;lt;/SsoBindingValidationLevel&amp;gt;
    &amp;lt;/Component&amp;gt;
    &amp;lt;Component xsi:type="ProtocolAuditComponent"&amp;gt;
      &amp;lt;OAuthClientId&amp;gt;N/A&amp;lt;/OAuthClientId&amp;gt;
      &amp;lt;OAuthGrant&amp;gt;N/A&amp;lt;/OAuthGrant&amp;gt;
    &amp;lt;/Component&amp;gt;
    &amp;lt;Component xsi:type="RequestAuditComponent"&amp;gt;
      &amp;lt;Server&amp;gt;Server&amp;lt;/Server&amp;gt;
      &amp;lt;AuthProtocol&amp;gt;SAMLP&amp;lt;/AuthProtocol&amp;gt;
      &amp;lt;NetworkLocation&amp;gt;Intranet&amp;lt;/NetworkLocation&amp;gt;
      &amp;lt;IpAddress&amp;gt;IpAddress&amp;lt;/IpAddress&amp;gt;
      &amp;lt;ForwardedIpAddress /&amp;gt;
      &amp;lt;ProxyIpAddress&amp;gt;N/A&amp;lt;/ProxyIpAddress&amp;gt;
      &amp;lt;NetworkIpAddress&amp;gt;N/A&amp;lt;/NetworkIpAddress&amp;gt;
      &amp;lt;ProxyServer&amp;gt;N/A&amp;lt;/ProxyServer&amp;gt;
      &amp;lt;UserAgentString&amp;gt;Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko&amp;lt;/UserAgentString&amp;gt;
      &amp;lt;Endpoint&amp;gt;/adfs/ls/wia&amp;lt;/Endpoint&amp;gt;
    &amp;lt;/Component&amp;gt;
  &amp;lt;/ContextComponents&amp;gt;
&amp;lt;/AuditBase&amp;gt;</Data>
    </EventData>
</Event>

[_joe]

Option 1:

# Would be good to apply only to the ADFS host if possible

# props.conf
[source::adfs-host]
REPORT-windows_broken_json_1200   = windows_broken_json

# transforms.conf 

[windows_broken_json]

FORMAT = $1::$2

REGEX = <(?!(?:headerName|headerValue))([^&/\=]+)>([^&]+)<

Option 2:
# Props.conf
[source::adfs-host]

EXTRACT-windows_activity_id       = \<\/System\>\<EventData\>\<Data\>(?<activity_id>[a-fA-F0-9\-]+)\<\/Data\>

EXTRACT-windows_adfs_src          = &lt;IpAddress&gt;(?<src>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})&lt;/IpAddress&gt;

EXTRACT-windows_adfs_user_id      = &lt;UserId&gt;(?<User_ID>[^&]+)&lt;/UserId&gt;

EXTRACT-windows_adfs_Target_User  = &lt;UserId&gt;([A-Za-z]{1,5}[\/\\]){0,1}(?<Target_User_Name>[^&]+)&lt;/UserId&gt;

EXTRACT-windows_adfs_AuditResult  = &lt;AuditResult&gt;(?<AuditResult>[A-Za-z]+)&lt;/AuditResult&gt;

You probably also want to add a lookup with definitions, etc, for these events since they aren't in the Splunk Windows TA.

Could also update your windows_activity_id to the following if you have 40x events which change the order around...

EXTRACT-windows_activity_id_1 = </System><EventData><Data>(?<instance_id>[a-f0-9]{2,20}-[a-f0-9]{2,20}-[a-f0-9]{2,20}-[a-f0-9]{2,20}-[a-f0-9]{2,20})</Data><Data>(?<activity_id>[a-f0-9]{2,20}-[a-f0-9]{2,20}-[a-f0-9]{2,20}-[a-f0-9]{2,20}-[a-f0-9]{2,20})</Data>

EXTRACT-windows_activity_id_2 = </System><EventData><Data>(?<activity_id>[a-f0-9]{2,20}-[a-f0-9]{2,20}-[a-f0-9]{2,20}-[a-f0-9]{2,20}-[a-f0-9]{2,20})</Data>

I would also love to know if there is a comprehensive CIM-compliant app for Windows ADFS logs.

Using adfs logs - https://www.splunk.com/en_us/blog/security/a-golden-saml-journey-solarwinds-continued.html

[lbruhns]

I have an admin on demand ticket open for this right now, did you ever get resolution?

[JonD]

I'm running into the same problem.  Was there a resolution to this?

[Ole]

I have the same problem. Did you find a solution?

[vinz2020]

I have the same problem.
Only on ADFS audit logs, maybe something to change on the windows server directly instead of touching splunk here?

[493669]

@samsonusmc ,
Use spath command to extarct field-

...|spath

https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Spath

[Meett]

This Search can be used to check data correctly during search time:
SPL:- index= <index_name> source=<source> host="<host>" sourcetype="XmlWinEventLog"
| rex mode=sed field=_raw "s/&lt;/</g"
| rex mode=sed field=_raw "s/&gt;/>/g"