Splunk Enterprise Security

How to parse ADFS Authentication logs for CIM compliance: Mixed XML and KV extraction

stroud_bc
Path Finder

We utilize Microsoft Active Directory Federation Services for SSO integration with several cloud applications. We would like the authentication events parsed by ADFS to be mapped to the Authentication data model for use in Enterprise Security, but unfortunately, the bulk of the useful fields are not extracted by Splunk_TA_windows.

I was hopeful when reviewing the answer given to the question linked below, and have made excellent use of |xmlkv and |spath while exploring the data, but what I really need is for fields from BOTH the XML block AND the key=value definitions at the top to align to the Authentication data model (fields like EventCode and Message, but also IpAddress and UserId)

hxxp://answers.splunk[.]com/answers/526219/parsing-xml-log-files-1.html

11/25/2019 00:00:00 AM
LogName=Security
SourceName=AD FS Auditing
EventCode=1202
EventType=0
Type=Information
ComputerName=server-hostname.domain
User=gMSA-adfs$
Sid=S-1-1-11-1111111111-111111111-111111111-1111
SidType=1
TaskCategory=Printers
OpCode=Info
RecordNumber=11111111
Keywords=Audit Success, Classic
Message=The Federation Service validated a new credential. See XML for details. 

Activity ID: 00000000-0000-0000-0000-000000000000 

Additional Data 
XML: <?xml version="1.0" encoding="utf-16"?>
<AuditBase xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="FreshCredentialAudit">
  <AuditType>FreshCredentials</AuditType>
  <AuditResult>Success</AuditResult>
  <FailureType>None</FailureType>
  <ErrorCode>N/A</ErrorCode>
  <ContextComponents>
    <Component xsi:type="ResourceAuditComponent">
      <RelyingParty>N/A</RelyingParty>
      <ClaimsProvider>AD AUTHORITY</ClaimsProvider>
      <UserId>DOMAIN\userid</UserId>
    </Component>
    <Component xsi:type="AuthNAuditComponent">
      <PrimaryAuth>N/A</PrimaryAuth>
      <DeviceAuth>false</DeviceAuth>
      <DeviceId>N/A</DeviceId>
      <MfaPerformed>false</MfaPerformed>
      <MfaMethod>N/A</MfaMethod>
      <TokenBindingProvidedId>false</TokenBindingProvidedId>
      <TokenBindingReferredId>false</TokenBindingReferredId>
      <SsoBindingValidationLevel>NotSet</SsoBindingValidationLevel>
    </Component>
    <Component xsi:type="ProtocolAuditComponent">
      <OAuthClientId>N/A</OAuthClientId>
      <OAuthGrant>N/A</OAuthGrant>
    </Component>
    <Component xsi:type="RequestAuditComponent">
      <Server>http://url.of.adfs.server/adfs/services/trust</Server>
      <AuthProtocol>N/A</AuthProtocol>
      <NetworkLocation>Extranet</NetworkLocation>
      <IpAddress>1.2.3.4,5.6.7.8</IpAddress>
      <ForwardedIpAddress>1.2.3.4,5.6.7.8</ForwardedIpAddress>
      <ProxyIpAddress>N/A</ProxyIpAddress>
      <NetworkIpAddress>N/A</NetworkIpAddress>
      <ProxyServer>proxy_server_hostname</ProxyServer>
      <UserAgentString>user_agent_string_goes_here</UserAgentString>
      <Endpoint>/adfs/services/trust/2005/usernamemixed</Endpoint>
    </Component>
  </ContextComponents>
</AuditBase>
0 Karma
1 Solution

to4kawa
Ultra Champion
| makeresults
| eval _raw="11/25/2019 00:00:00 AM
 LogName=Security
 SourceName=AD FS Auditing
 EventCode=1202
 EventType=0
 Type=Information
 ComputerName=server-hostname.domain
 User=gMSA-adfs$
 Sid=S-1-1-11-1111111111-111111111-111111111-1111
 SidType=1
 TaskCategory=Printers
 OpCode=Info
 RecordNumber=11111111
 Keywords=Audit Success, Classic
 Message=The Federation Service validated a new credential. See XML for details. 

 Activity ID: 00000000-0000-0000-0000-000000000000 

 Additional Data 
 XML: <?xml version=\"1.0\" encoding=\"utf-16\"?>
 <AuditBase xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:type=\"FreshCredentialAudit\">
   <AuditType>FreshCredentials</AuditType>
   <AuditResult>Success</AuditResult>
   <FailureType>None</FailureType>
   <ErrorCode>N/A</ErrorCode>
   <ContextComponents>
     <Component xsi:type=\"ResourceAuditComponent\">
       <RelyingParty>N/A</RelyingParty>
       <ClaimsProvider>AD AUTHORITY</ClaimsProvider>
       <UserId>DOMAIN\userid</UserId>
     </Component>
     <Component xsi:type=\"AuthNAuditComponent\">
       <PrimaryAuth>N/A</PrimaryAuth>
       <DeviceAuth>false</DeviceAuth>
       <DeviceId>N/A</DeviceId>
       <MfaPerformed>false</MfaPerformed>
       <MfaMethod>N/A</MfaMethod>
       <TokenBindingProvidedId>false</TokenBindingProvidedId>
       <TokenBindingReferredId>false</TokenBindingReferredId>
       <SsoBindingValidationLevel>NotSet</SsoBindingValidationLevel>
     </Component>
     <Component xsi:type=\"ProtocolAuditComponent\">
       <OAuthClientId>N/A</OAuthClientId>
       <OAuthGrant>N/A</OAuthGrant>
     </Component>
     <Component xsi:type=\"RequestAuditComponent\">
       <Server>http://url.of.adfs.server/adfs/services/trust</Server>
       <AuthProtocol>N/A</AuthProtocol>
       <NetworkLocation>Extranet</NetworkLocation>
       <IpAddress>1.2.3.4,5.6.7.8</IpAddress>
       <ForwardedIpAddress>1.2.3.4,5.6.7.8</ForwardedIpAddress>
       <ProxyIpAddress>N/A</ProxyIpAddress>
       <NetworkIpAddress>N/A</NetworkIpAddress>
       <ProxyServer>proxy_server_hostname</ProxyServer>
       <UserAgentString>user_agent_string_goes_here</UserAgentString>
       <Endpoint>/adfs/services/trust/2005/usernamemixed</Endpoint>
     </Component>
   </ContextComponents>
 </AuditBase>"
 | rex field=_raw "(?s)XML: (?<xml_data>.+)"
 | spath input=xml_data
 | kv
 | rex field=_raw "Activity ID: (?<Activity_ID>.+)"

I managed to get the field.

View solution in original post

to4kawa
Ultra Champion

hi @jak3d84 

my query will run on multiple logs .

if your log is XML , it is OK.

please check command reference.

0 Karma

to4kawa
Ultra Champion
| makeresults
| eval _raw="11/25/2019 00:00:00 AM
 LogName=Security
 SourceName=AD FS Auditing
 EventCode=1202
 EventType=0
 Type=Information
 ComputerName=server-hostname.domain
 User=gMSA-adfs$
 Sid=S-1-1-11-1111111111-111111111-111111111-1111
 SidType=1
 TaskCategory=Printers
 OpCode=Info
 RecordNumber=11111111
 Keywords=Audit Success, Classic
 Message=The Federation Service validated a new credential. See XML for details. 

 Activity ID: 00000000-0000-0000-0000-000000000000 

 Additional Data 
 XML: <?xml version=\"1.0\" encoding=\"utf-16\"?>
 <AuditBase xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:type=\"FreshCredentialAudit\">
   <AuditType>FreshCredentials</AuditType>
   <AuditResult>Success</AuditResult>
   <FailureType>None</FailureType>
   <ErrorCode>N/A</ErrorCode>
   <ContextComponents>
     <Component xsi:type=\"ResourceAuditComponent\">
       <RelyingParty>N/A</RelyingParty>
       <ClaimsProvider>AD AUTHORITY</ClaimsProvider>
       <UserId>DOMAIN\userid</UserId>
     </Component>
     <Component xsi:type=\"AuthNAuditComponent\">
       <PrimaryAuth>N/A</PrimaryAuth>
       <DeviceAuth>false</DeviceAuth>
       <DeviceId>N/A</DeviceId>
       <MfaPerformed>false</MfaPerformed>
       <MfaMethod>N/A</MfaMethod>
       <TokenBindingProvidedId>false</TokenBindingProvidedId>
       <TokenBindingReferredId>false</TokenBindingReferredId>
       <SsoBindingValidationLevel>NotSet</SsoBindingValidationLevel>
     </Component>
     <Component xsi:type=\"ProtocolAuditComponent\">
       <OAuthClientId>N/A</OAuthClientId>
       <OAuthGrant>N/A</OAuthGrant>
     </Component>
     <Component xsi:type=\"RequestAuditComponent\">
       <Server>http://url.of.adfs.server/adfs/services/trust</Server>
       <AuthProtocol>N/A</AuthProtocol>
       <NetworkLocation>Extranet</NetworkLocation>
       <IpAddress>1.2.3.4,5.6.7.8</IpAddress>
       <ForwardedIpAddress>1.2.3.4,5.6.7.8</ForwardedIpAddress>
       <ProxyIpAddress>N/A</ProxyIpAddress>
       <NetworkIpAddress>N/A</NetworkIpAddress>
       <ProxyServer>proxy_server_hostname</ProxyServer>
       <UserAgentString>user_agent_string_goes_here</UserAgentString>
       <Endpoint>/adfs/services/trust/2005/usernamemixed</Endpoint>
     </Component>
   </ContextComponents>
 </AuditBase>"
 | rex field=_raw "(?s)XML: (?<xml_data>.+)"
 | spath input=xml_data
 | kv
 | rex field=_raw "Activity ID: (?<Activity_ID>.+)"

I managed to get the field.

stroud_bc
Path Finder

You're the man! It hadn't even occurred to me to rex the XML out and treat it separately-- thanks!

0 Karma

jak3d84
Loves-to-Learn Lots

Hey There!

So we are running into something similar with ADFS logs not being CIM compliant. Can you explain to me what your reasoning was for formatting your search below? Also, is this specific to one log? How would I look at multiple logs with this search?

Thanks,

Jake

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...