Getting Data In

Why won't Splunk parse my multi-line event properly?

smcdonald20
Path Finder

I am currently unable to parse my multi-line event properly using Splunk.
Here is an example from the start of the event:

<?xml version="1.0" encoding="utf-16"?>

<report>

<GPO xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.microsoft.com/GroupPolicy/Settings">
      <Identifier>
        <Identifier xmlns="http://www.microsoft.com/GroupPolicy/Types">{31B2F340-016D-11D2-945F-00C04FB984F9}</Identifier>
        <Domain xmlns="http://www.microsoft.com/GroupPolicy/Types">options-it.com</Domain>
      </Identifier>
      <Name>Default Domain Policy</Name>
      <IncludeComments>true</IncludeComments>
      <CreatedTime>2002-09-17T07:41:34</CreatedTime>
      <ModifiedTime>2018-05-03T13:58:32</ModifiedTime>
      <ReadTime>2018-07-09T04:00:36.6876121Z</ReadTime>
      <SecurityDescriptor>
        <SDDL xmlns="http://www.microsoft.com/GroupPolicy/Types/Security">O:DAG:DAD:PAI(OA;CI;CR;edacfd8f-ffb3-11d1-b41d-00a0c968f939;;AU)(A;;CCDCLCSWRPWPDTLOSDRCWDWO;;;DA)(A;CI;LCRPRC;;;S-1-5-21-1060284298-1275210071-1417001333-95787)(A;CI;LCRPRC;;;S-1-5-21-1060284298-1275210071-1417001333-12472)(A;CI;CCDCLCRPWPSDRCWDWO;;;S-1-5-21-1060284298-1275210071-1417001333-95786)(A;CI;CCDCLCRPWPSDRCWDWO;;;S-1-5-21-1060284298-1275210071-1417001333-22697)(A;CI;CCDCLCSWRPWPDTLOSDRCWDWO;;;DA)(A;CI;CCDCLCSWRPWPDTLOSDRCWDWO;;;S-1-5-21-1060284298-1275210071-1417001333-519)(A;;LCRPLORC;;;ED)(A;CI;LCRPLORC;;;AU)(A;CI;CCDCLCSWRPWPDTLOSDRCWDWO;;;SY)(A;CIIO;CCDCLCSWRPWPDTLOSDRCWDWO;;;CO)S:AI(AU;CIIDSA;CCDCSWWPDTLOCRSDWDWO;;;WD)(AU;CIIDFA;CCDCSWWPDTCRSDWDWO;;;WD)</SDDL>
        <Owner xmlns="http://www.microsoft.com/GroupPolicy/Types/Security">
          <SID xmlns="http://www.microsoft.com/GroupPolicy/Types">S-1-5-21-1060284298-1275210071-1417001333-512</SID>
          <Name xmlns="http://www.microsoft.com/GroupPolicy/Types">OPTIONS-IT\Domain Admins</Name>
        </Owner>

I am trying to get it split the events properly, where each event starts with this line:

<GPO xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.microsoft.com/GroupPolicy/Settings">

This is the props settings im trying:

    BREAK_ONLY_BEFORE=.+GPO\sxmlns:xsd.+
    CHARSET=UTF-16LE
    SHOULD_LINEMERGE=false
    disabled=false
    TIME_FORMAT=%Y-%m-%dT%H:%M:%S
    TIME_PREFIX=.+<ReadTime>
    MAX_TIMESTAMP=18
    LINE_BREAKER=.+GPO\sxmlns:xsd.+
0 Karma
1 Solution

FrankVl
Ultra Champion

Couple of comments:

  • I don't think you should mix "break only before" and "line breaker" in 1 props.conf.
  • Line Breaker should have a capturing group (usually the line ending before the start of the event).
  • don't add those .+ around time prefix and line breaker, no need for it and especially in the line breaker case it completely defeats the purpose of that setting
  • TIME_PREFIX is a regex, so <> characters need to be escaped.
  • I guess you meant MAX_TIMESTAMP_LOOKAHEAD not MAX_TIMESTAMP.

Can you try this:

 CHARSET=UTF-16LE
 SHOULD_LINEMERGE=false
 disabled=false
 TIME_FORMAT=%Y-%m-%dT%H:%M:%S
 TIME_PREFIX=\<ReadTime\>
 MAX_TIMESTAMP_LOOKAHEAD=18
 LINE_BREAKER=([\r\n]+)\<GPO\sxmlns:xsd

View solution in original post

0 Karma

FrankVl
Ultra Champion

Couple of comments:

  • I don't think you should mix "break only before" and "line breaker" in 1 props.conf.
  • Line Breaker should have a capturing group (usually the line ending before the start of the event).
  • don't add those .+ around time prefix and line breaker, no need for it and especially in the line breaker case it completely defeats the purpose of that setting
  • TIME_PREFIX is a regex, so <> characters need to be escaped.
  • I guess you meant MAX_TIMESTAMP_LOOKAHEAD not MAX_TIMESTAMP.

Can you try this:

 CHARSET=UTF-16LE
 SHOULD_LINEMERGE=false
 disabled=false
 TIME_FORMAT=%Y-%m-%dT%H:%M:%S
 TIME_PREFIX=\<ReadTime\>
 MAX_TIMESTAMP_LOOKAHEAD=18
 LINE_BREAKER=([\r\n]+)\<GPO\sxmlns:xsd

View solution in original post

0 Karma

smcdonald20
Path Finder

Thanks Frank!
This changes to capture EVERYTHING as one event, doesn't seem to be breaking at the

0 Karma

FrankVl
Ultra Champion

Your sample file contains a space before the <GPO... tag. Is that also there in the actual data? If so, you need to add change the line breaker to: ([\r\n]+\s+)\<GPO\sxmlns:xsd

0 Karma

smcdonald20
Path Finder

Your original change worked, i accidentally copied the "=" from the Line Breaker!
Thank you very much!

0 Karma

FrankVl
Ultra Champion

Please put your example start line also as code, otherwise it disappears due to how the board software handles <> characters.

And please post any relevant props.conf settings your tried so far.

0 Karma

smcdonald20
Path Finder

Thanks Frank, please see updates! any help appreciated!

0 Karma