How to parse ADFS Authentication logs for CIM compliance: Mixed XML and KV extraction

[stroud_bc]

We utilize Microsoft Active Directory Federation Services for SSO integration with several cloud applications. We would like the authentication events parsed by ADFS to be mapped to the Authentication data model for use in Enterprise Security, but unfortunately, the bulk of the useful fields are not extracted by Splunk_TA_windows.

I was hopeful when reviewing the answer given to the question linked below, and have made excellent use of |xmlkv and |spath while exploring the data, but what I really need is for fields from BOTH the XML block AND the key=value definitions at the top to align to the Authentication data model (fields like EventCode and Message, but also IpAddress and UserId)

hxxp://answers.splunk[.]com/answers/526219/parsing-xml-log-files-1.html

11/25/2019 00:00:00 AM
LogName=Security
SourceName=AD FS Auditing
EventCode=1202
EventType=0
Type=Information
ComputerName=server-hostname.domain
User=gMSA-adfs$
Sid=S-1-1-11-1111111111-111111111-111111111-1111
SidType=1
TaskCategory=Printers
OpCode=Info
RecordNumber=11111111
Keywords=Audit Success, Classic
Message=The Federation Service validated a new credential. See XML for details. 

Activity ID: 00000000-0000-0000-0000-000000000000 

Additional Data 
XML: <?xml version="1.0" encoding="utf-16"?>
<AuditBase xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="FreshCredentialAudit">
  <AuditType>FreshCredentials</AuditType>
  <AuditResult>Success</AuditResult>
  <FailureType>None</FailureType>
  <ErrorCode>N/A</ErrorCode>
  <ContextComponents>
    <Component xsi:type="ResourceAuditComponent">
      <RelyingParty>N/A</RelyingParty>
      <ClaimsProvider>AD AUTHORITY</ClaimsProvider>
      <UserId>DOMAIN\userid</UserId>
    </Component>
    <Component xsi:type="AuthNAuditComponent">
      <PrimaryAuth>N/A</PrimaryAuth>
      <DeviceAuth>false</DeviceAuth>
      <DeviceId>N/A</DeviceId>
      <MfaPerformed>false</MfaPerformed>
      <MfaMethod>N/A</MfaMethod>
      <TokenBindingProvidedId>false</TokenBindingProvidedId>
      <TokenBindingReferredId>false</TokenBindingReferredId>
      <SsoBindingValidationLevel>NotSet</SsoBindingValidationLevel>
    </Component>
    <Component xsi:type="ProtocolAuditComponent">
      <OAuthClientId>N/A</OAuthClientId>
      <OAuthGrant>N/A</OAuthGrant>
    </Component>
    <Component xsi:type="RequestAuditComponent">
      <Server>http://url.of.adfs.server/adfs/services/trust</Server>
      <AuthProtocol>N/A</AuthProtocol>
      <NetworkLocation>Extranet</NetworkLocation>
      <IpAddress>1.2.3.4,5.6.7.8</IpAddress>
      <ForwardedIpAddress>1.2.3.4,5.6.7.8</ForwardedIpAddress>
      <ProxyIpAddress>N/A</ProxyIpAddress>
      <NetworkIpAddress>N/A</NetworkIpAddress>
      <ProxyServer>proxy_server_hostname</ProxyServer>
      <UserAgentString>user_agent_string_goes_here</UserAgentString>
      <Endpoint>/adfs/services/trust/2005/usernamemixed</Endpoint>
    </Component>
  </ContextComponents>
</AuditBase>

[to4kawa]

hi @jak3d84 

my query will run on multiple logs .

if your log is XML , it is OK.

please check command reference.

[to4kawa · Accepted solution]

| makeresults
| eval _raw="11/25/2019 00:00:00 AM
 LogName=Security
 SourceName=AD FS Auditing
 EventCode=1202
 EventType=0
 Type=Information
 ComputerName=server-hostname.domain
 User=gMSA-adfs$
 Sid=S-1-1-11-1111111111-111111111-111111111-1111
 SidType=1
 TaskCategory=Printers
 OpCode=Info
 RecordNumber=11111111
 Keywords=Audit Success, Classic
 Message=The Federation Service validated a new credential. See XML for details. 

 Activity ID: 00000000-0000-0000-0000-000000000000 

 Additional Data 
 XML: <?xml version=\"1.0\" encoding=\"utf-16\"?>
 <AuditBase xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:type=\"FreshCredentialAudit\">
   <AuditType>FreshCredentials</AuditType>
   <AuditResult>Success</AuditResult>
   <FailureType>None</FailureType>
   <ErrorCode>N/A</ErrorCode>
   <ContextComponents>
     <Component xsi:type=\"ResourceAuditComponent\">
       <RelyingParty>N/A</RelyingParty>
       <ClaimsProvider>AD AUTHORITY</ClaimsProvider>
       <UserId>DOMAIN\userid</UserId>
     </Component>
     <Component xsi:type=\"AuthNAuditComponent\">
       <PrimaryAuth>N/A</PrimaryAuth>
       <DeviceAuth>false</DeviceAuth>
       <DeviceId>N/A</DeviceId>
       <MfaPerformed>false</MfaPerformed>
       <MfaMethod>N/A</MfaMethod>
       <TokenBindingProvidedId>false</TokenBindingProvidedId>
       <TokenBindingReferredId>false</TokenBindingReferredId>
       <SsoBindingValidationLevel>NotSet</SsoBindingValidationLevel>
     </Component>
     <Component xsi:type=\"ProtocolAuditComponent\">
       <OAuthClientId>N/A</OAuthClientId>
       <OAuthGrant>N/A</OAuthGrant>
     </Component>
     <Component xsi:type=\"RequestAuditComponent\">
       <Server>http://url.of.adfs.server/adfs/services/trust</Server>
       <AuthProtocol>N/A</AuthProtocol>
       <NetworkLocation>Extranet</NetworkLocation>
       <IpAddress>1.2.3.4,5.6.7.8</IpAddress>
       <ForwardedIpAddress>1.2.3.4,5.6.7.8</ForwardedIpAddress>
       <ProxyIpAddress>N/A</ProxyIpAddress>
       <NetworkIpAddress>N/A</NetworkIpAddress>
       <ProxyServer>proxy_server_hostname</ProxyServer>
       <UserAgentString>user_agent_string_goes_here</UserAgentString>
       <Endpoint>/adfs/services/trust/2005/usernamemixed</Endpoint>
     </Component>
   </ContextComponents>
 </AuditBase>"
 | rex field=_raw "(?s)XML: (?<xml_data>.+)"
 | spath input=xml_data
 | kv
 | rex field=_raw "Activity ID: (?<Activity_ID>.+)"

I managed to get the field.

[stroud_bc]

You're the man! It hadn't even occurred to me to rex the XML out and treat it separately-- thanks!

[jak3d84]

Hey There!

So we are running into something similar with ADFS logs not being CIM compliant. Can you explain to me what your reasoning was for formatting your search below? Also, is this specific to one log? How would I look at multiple logs with this search?

Thanks,

Jake