Using regex capture group on a line_breaker attribute for XML parsing of GPO Logs

smcdonald20 — Tue, 29 Sep 2020 15:54:03 GMT

I have an XML file, with information regarding Windows GPOs.

Each Event Looks like the below.
The issue is, it is not breaking the data correctly.
How should i change the props.conf file so that it always breaks after the last line ""

I have tried this, but doesn't work (have also tried LINE_BREAKER and also doesn't work):
[source::mysource]
DATETIME_CONFIG = CURRENT
KV_MODE = xml
SHOULD_LINEMERGE =true
MUST_BREAK_AFTER = <\/GPO>

<GPO xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.microsoft.com/GroupPolicy/Settings">
  <Identifier>
    <Identifier xmlns="http://www.microsoft.com/GroupPolicy/Types">{3ABBC184-C943-44B8-8C85-EDB5A8AC6E91}</Identifier>
    <Domain xmlns="http://www.microsoft.com/GroupPolicy/Types">options-it.com</Domain>
  </Identifier>
  <Name>Northbay Service Accounts</Name>
  <IncludeComments>true</IncludeComments>
  <CreatedTime>2005-11-15T22:46:05</CreatedTime>
  <ModifiedTime>2017-09-02T02:00:13</ModifiedTime>
  <ReadTime>2017-09-19T12:35:06.845031Z</ReadTime>
  <SecurityDescriptor>
    <SDDL xmlns="http://www.microsoft.com/GroupPolicy/Types/Security">O:DAG:DUD:PAI(OA;CI;CR;edacfd8f-ffb3-11d1-b41d-00a0c968f939;;AU)(A;;CCDCLCSWRPWPDTLOSDRCWDWO;;;DA)(A;CI;LCRPRC;;;S-1-5-21-1060284298-1275210071-1417001333-12472)(A;CI;CCDCLCRPWPSDRCWDWO;;;S-1-5-21-1060284298-1275210071-1417001333-22697)(A;CI;CCDCLCSWRPWPDTLOSDRCWDWO;;;DA)(A;CI;CCDCLCSWRPWPDTLOSDRCWDWO;;;S-1-5-21-1060284298-1275210071-1417001333-519)(A;CI;LCRPLORC;;;ED)(A;CI;LCRPLORC;;;AU)(A;CI;CCDCLCSWRPWPDTLOSDRCWDWO;;;SY)(A;CIIO;CCDCLCSWRPWPDTLOSDRCWDWO;;;CO)S:AI(AU;CIIDSA;CCDCSWWPDTLOCRSDWDWO;;;WD)(AU;CIIDFA;CCDCSWWPDTCRSDWDWO;;;WD)</SDDL>
    <Owner xmlns="http://www.microsoft.com/GroupPolicy/Types/Security">
      <SID xmlns="http://www.microsoft.com/GroupPolicy/Types">S-1-5-21-1060284298-1275210071-1417001333-512</SID>
      <Name xmlns="http://www.microsoft.com/GroupPolicy/Types">OPTIONS-IT\Domain Admins</Name>
    </Owner>
    <Group xmlns="http://www.microsoft.com/GroupPolicy/Types/Security">
      <SID xmlns="http://www.microsoft.com/GroupPolicy/Types">S-1-5-21-1060284298-1275210071-1417001333-513</SID>
      <Name xmlns="http://www.microsoft.com/GroupPolicy/Types">OPTIONS-IT\Domain Users</Name>
    </Group>
    <PermissionsPresent xmlns="http://www.microsoft.com/GroupPolicy/Types/Security">true</PermissionsPresent>
    <Permissions xmlns="http://www.microsoft.com/GroupPolicy/Types/Security">
      <InheritsFromParent>false</InheritsFromParent>
      <TrusteePermissions>
        <Trustee>
          <SID xmlns="http://www.microsoft.com/GroupPolicy/Types">S-1-5-21-1060284298-1275210071-1417001333-512</SID>
          <Name xmlns="http://www.microsoft.com/GroupPolicy/Types">OPTIONS-IT\Domain Admins</Name>
        </Trustee>
        <Type xsi:type="PermissionType">
          <PermissionType>Allow</PermissionType>
        </Type>
        <Inherited>false</Inherited>
        <Applicability>
          <ToSelf>true</ToSelf>
          <ToDescendantObjects>false</ToDescendantObjects>
          <ToDescendantContainers>true</ToDescendantContainers>
          <ToDirectDescendantsOnly>false</ToDirectDescendantsOnly>
        </Applicability>
        <Standard>
          <GPOGroupedAccessEnum>Edit, delete, modify security</GPOGroupedAccessEnum>
        </Standard>
        <AccessMask>0</AccessMask>
      </TrusteePermissions>
      <TrusteePermissions>
        <Trustee>
          <SID xmlns="http://www.microsoft.com/GroupPolicy/Types">S-1-5-9</SID>
          <Name xmlns="http://www.microsoft.com/GroupPolicy/Types">NT AUTHORITY\ENTERPRISE DOMAIN CONTROLLERS</Name>
        </Trustee>
        <Type xsi:type="PermissionType">
          <PermissionType>Allow</PermissionType>
        </Type>
        <Inherited>false</Inherited>
        <Applicability>
          <ToSelf>true</ToSelf>
          <ToDescendantObjects>false</ToDescendantObjects>
          <ToDescendantContainers>true</ToDescendantContainers>
          <ToDirectDescendantsOnly>false</ToDirectDescendantsOnly>
        </Applicability>
        <Standard>
          <GPOGroupedAccessEnum>Read</GPOGroupedAccessEnum>
        </Standard>
        <AccessMask>0</AccessMask>
      </TrusteePermissions>
      <TrusteePermissions>
        <Trustee>
          <SID xmlns="http://www.microsoft.com/GroupPolicy/Types">S-1-5-18</SID>
          <Name xmlns="http://www.microsoft.com/GroupPolicy/Types">NT AUTHORITY\SYSTEM</Name>
        </Trustee>
        <Type xsi:type="PermissionType">
          <PermissionType>Allow</PermissionType>
        </Type>
        <Inherited>false</Inherited>
        <Applicability>
          <ToSelf>true</ToSelf>
          <ToDescendantObjects>false</ToDescendantObjects>
          <ToDescendantContainers>true</ToDescendantContainers>
          <ToDirectDescendantsOnly>false</ToDirectDescendantsOnly>
        </Applicability>
        <Standard>
          <GPOGroupedAccessEnum>Edit, delete, modify security</GPOGroupedAccessEnum>
        </Standard>
        <AccessMask>0</AccessMask>
      </TrusteePermissions>
      <TrusteePermissions>
        <Trustee>
          <SID xmlns="http://www.microsoft.com/GroupPolicy/Types">S-1-5-21-1060284298-1275210071-1417001333-519</SID>
          <Name xmlns="http://www.microsoft.com/GroupPolicy/Types">OPTIONS-IT\Enterprise Admins</Name>
        </Trustee>
        <Type xsi:type="PermissionType">
          <PermissionType>Allow</PermissionType>
        </Type>
        <Inherited>false</Inherited>
        <Applicability>
          <ToSelf>true</ToSelf>
          <ToDescendantObjects>false</ToDescendantObjects>
          <ToDescendantContainers>true</ToDescendantContainers>
          <ToDirectDescendantsOnly>false</ToDirectDescendantsOnly>
        </Applicability>
        <Standard>
          <GPOGroupedAccessEnum>Edit, delete, modify security</GPOGroupedAccessEnum>
        </Standard>
        <AccessMask>0</AccessMask>
      </TrusteePermissions>
      <TrusteePermissions>
        <Trustee>
          <SID xmlns="http://www.microsoft.com/GroupPolicy/Types">S-1-5-21-1060284298-1275210071-1417001333-12472</SID>
          <Name xmlns="http://www.microsoft.com/GroupPolicy/Types">OPTIONS-IT\Admins PIPE Domain</Name>
        </Trustee>
        <Type xsi:type="PermissionType">
          <PermissionType>Allow</PermissionType>
        </Type>
        <Inherited>false</Inherited>
        <Applicability>
          <ToSelf>true</ToSelf>
          <ToDescendantObjects>false</ToDescendantObjects>
          <ToDescendantContainers>true</ToDescendantContainers>
          <ToDirectDescendantsOnly>false</ToDirectDescendantsOnly>
        </Applicability>
        <Standard>
          <GPOGroupedAccessEnum>Read</GPOGroupedAccessEnum>
        </Standard>
        <AccessMask>0</AccessMask>
      </TrusteePermissions>
      <TrusteePermissions>
        <Trustee>
          <SID xmlns="http://www.microsoft.com/GroupPolicy/Types">S-1-5-11</SID>
          <Name xmlns="http://www.microsoft.com/GroupPolicy/Types">NT AUTHORITY\Authenticated Users</Name>
        </Trustee>
        <Type xsi:type="PermissionType">
          <PermissionType>Allow</PermissionType>
        </Type>
        <Inherited>false</Inherited>
        <Applicability>
          <ToSelf>true</ToSelf>
          <ToDescendantObjects>false</ToDescendantObjects>
          <ToDescendantContainers>true</ToDescendantContainers>
          <ToDirectDescendantsOnly>false</ToDirectDescendantsOnly>
        </Applicability>
        <Standard>
          <GPOGroupedAccessEnum>Apply Group Policy</GPOGroupedAccessEnum>
        </Standard>
        <AccessMask>0</AccessMask>
      </TrusteePermissions>
      <TrusteePermissions>
        <Trustee>
          <SID xmlns="http://www.microsoft.com/GroupPolicy/Types">S-1-5-21-1060284298-1275210071-1417001333-22697</SID>
          <Name xmlns="http://www.microsoft.com/GroupPolicy/Types">OPTIONS-IT\Admins PIPE Systems Engineers</Name>
        </Trustee>
        <Type xsi:type="PermissionType">
          <PermissionType>Allow</PermissionType>
        </Type>
        <Inherited>false</Inherited>
        <Applicability>
          <ToSelf>true</ToSelf>
          <ToDescendantObjects>false</ToDescendantObjects>
          <ToDescendantContainers>true</ToDescendantContainers>
          <ToDirectDescendantsOnly>false</ToDirectDescendantsOnly>
        </Applicability>
        <Standard>
          <GPOGroupedAccessEnum>Edit, delete, modify security</GPOGroupedAccessEnum>
        </Standard>
        <AccessMask>0</AccessMask>
      </TrusteePermissions>
    </Permissions>
    <AuditingPresent xmlns="http://www.microsoft.com/GroupPolicy/Types/Security">false</AuditingPresent>
  </SecurityDescriptor>
  <FilterDataAvailable>true</FilterDataAvailable>
  <Computer>
    <VersionDirectory>6</VersionDirectory>
    <VersionSysvol>6</VersionSysvol>
    <Enabled>true</Enabled>
    <ExtensionData>
      <Extension xmlns:q1="http://www.microsoft.com/GroupPolicy/Settings/Security" xsi:type="q1:SecuritySettings">
        <q1:Account>
          <q1:Name>LockoutBadCount</q1:Name>
          <q1:SettingNumber>0</q1:SettingNumber>
          <q1:Type>Account Lockout</q1:Type>
        </q1:Account>
      </Extension>
      <Name>Security</Name>
    </ExtensionData>
  </Computer>
  <User>
    <VersionDirectory>0</VersionDirectory>
    <VersionSysvol>0</VersionSysvol>
    <Enabled>true</Enabled>
  </User>
</GPO>

Re: Using regex capture group on a line_breaker attribute for XML parsing of GPO Logs

richgalloway — Thu, 21 Sep 2017 13:04:35 GMT

The LINE_BREAKER attribute needs a regex capture group. Try LINE_BREAKER = \/GPO>().

Re: Using regex capture group on a line_breaker attribute for XML parsing of GPO Logs

DMohn — Thu, 21 Sep 2017 13:07:27 GMT

Hi @smcdonald20

Try this setting:

[your_sourcetype]
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE=<GPO\s
KV_MODE=xml
TRUNCATE=999999
DATETIME_CONFIG=CURRENT

Re: Using regex capture group on a line_breaker attribute for XML parsing of GPO Logs

smcdonald20 — Thu, 21 Sep 2017 13:31:38 GMT

Thanks, but this doesn't seem to have worked.
Can you confirm if the only thing should need to do is edit the props.conf file and restart splunk?

Are there any other config files i should be changing?

Re: Using regex capture group on a line_breaker attribute for XML parsing of GPO Logs

DMohn — Thu, 21 Sep 2017 13:49:32 GMT

You have to reingest the data, as the extraction happens on index time (you are using indexed extractions) ...

Re: Using regex capture group on a line_breaker attribute for XML parsing of GPO Logs

smcdonald20 — Thu, 21 Sep 2017 13:53:06 GMT

Thank you! this worked!!

topic Re: Using regex capture group on a line_breaker attribute for XML parsing of GPO Logs in Splunk Search

Using regex capture group on a line_breaker attribute for XML parsing of GPO Logs

Re: Using regex capture group on a line_breaker attribute for XML parsing of GPO Logs

Re: Using regex capture group on a line_breaker attribute for XML parsing of GPO Logs

Re: Using regex capture group on a line_breaker attribute for XML parsing of GPO Logs

Re: Using regex capture group on a line_breaker attribute for XML parsing of GPO Logs

Re: Using regex capture group on a line_breaker attribute for XML parsing of GPO Logs