topic Why won't Splunk parse my multi-line event properly? in Getting Data In

Why won't Splunk parse my multi-line event properly?

smcdonald20 — Mon, 09 Jul 2018 11:45:20 GMT

I am currently unable to parse my multi-line event properly using Splunk.
Here is an example from the start of the event:

<?xml version="1.0" encoding="utf-16"?>

<report>

<GPO xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.microsoft.com/GroupPolicy/Settings">
      <Identifier>
        <Identifier xmlns="http://www.microsoft.com/GroupPolicy/Types">{31B2F340-016D-11D2-945F-00C04FB984F9}</Identifier>
        <Domain xmlns="http://www.microsoft.com/GroupPolicy/Types">options-it.com</Domain>
      </Identifier>
      <Name>Default Domain Policy</Name>
      <IncludeComments>true</IncludeComments>
      <CreatedTime>2002-09-17T07:41:34</CreatedTime>
      <ModifiedTime>2018-05-03T13:58:32</ModifiedTime>
      <ReadTime>2018-07-09T04:00:36.6876121Z</ReadTime>
      <SecurityDescriptor>
        <SDDL xmlns="http://www.microsoft.com/GroupPolicy/Types/Security">O:DAG:DAD:PAI(OA;CI;CR;edacfd8f-ffb3-11d1-b41d-00a0c968f939;;AU)(A;;CCDCLCSWRPWPDTLOSDRCWDWO;;;DA)(A;CI;LCRPRC;;;S-1-5-21-1060284298-1275210071-1417001333-95787)(A;CI;LCRPRC;;;S-1-5-21-1060284298-1275210071-1417001333-12472)(A;CI;CCDCLCRPWPSDRCWDWO;;;S-1-5-21-1060284298-1275210071-1417001333-95786)(A;CI;CCDCLCRPWPSDRCWDWO;;;S-1-5-21-1060284298-1275210071-1417001333-22697)(A;CI;CCDCLCSWRPWPDTLOSDRCWDWO;;;DA)(A;CI;CCDCLCSWRPWPDTLOSDRCWDWO;;;S-1-5-21-1060284298-1275210071-1417001333-519)(A;;LCRPLORC;;;ED)(A;CI;LCRPLORC;;;AU)(A;CI;CCDCLCSWRPWPDTLOSDRCWDWO;;;SY)(A;CIIO;CCDCLCSWRPWPDTLOSDRCWDWO;;;CO)S:AI(AU;CIIDSA;CCDCSWWPDTLOCRSDWDWO;;;WD)(AU;CIIDFA;CCDCSWWPDTCRSDWDWO;;;WD)</SDDL>
        <Owner xmlns="http://www.microsoft.com/GroupPolicy/Types/Security">
          <SID xmlns="http://www.microsoft.com/GroupPolicy/Types">S-1-5-21-1060284298-1275210071-1417001333-512</SID>
          <Name xmlns="http://www.microsoft.com/GroupPolicy/Types">OPTIONS-IT\Domain Admins</Name>
        </Owner>

I am trying to get it split the events properly, where each event starts with this line:

<GPO xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.microsoft.com/GroupPolicy/Settings">

This is the props settings im trying:

    BREAK_ONLY_BEFORE=.+GPO\sxmlns:xsd.+
    CHARSET=UTF-16LE
    SHOULD_LINEMERGE=false
    disabled=false
    TIME_FORMAT=%Y-%m-%dT%H:%M:%S
    TIME_PREFIX=.+<ReadTime>
    MAX_TIMESTAMP=18
    LINE_BREAKER=.+GPO\sxmlns:xsd.+

Re: Why won't Splunk parse my multi-line event properly?

FrankVl — Mon, 09 Jul 2018 11:58:51 GMT

Please put your example start line also as code, otherwise it disappears due to how the board software handles <> characters.

And please post any relevant props.conf settings your tried so far.

Re: Why won't Splunk parse my multi-line event properly?

smcdonald20 — Mon, 09 Jul 2018 12:12:36 GMT

Thanks Frank, please see updates! any help appreciated!

Re: Why won't Splunk parse my multi-line event properly?

FrankVl — Mon, 09 Jul 2018 12:18:33 GMT

Couple of comments:

I don't think you should mix "break only before" and "line breaker" in 1 props.conf.
Line Breaker should have a capturing group (usually the line ending before the start of the event).
don't add those .+ around time prefix and line breaker, no need for it and especially in the line breaker case it completely defeats the purpose of that setting
TIME_PREFIX is a regex, so <> characters need to be escaped.
I guess you meant MAX_TIMESTAMP_LOOKAHEAD not MAX_TIMESTAMP.

Can you try this:

 CHARSET=UTF-16LE
 SHOULD_LINEMERGE=false
 disabled=false
 TIME_FORMAT=%Y-%m-%dT%H:%M:%S
 TIME_PREFIX=\<ReadTime\>
 MAX_TIMESTAMP_LOOKAHEAD=18
 LINE_BREAKER=([\r\n]+)\<GPO\sxmlns:xsd

Re: Why won't Splunk parse my multi-line event properly?

smcdonald20 — Mon, 09 Jul 2018 12:25:50 GMT

Thanks Frank!
This changes to capture EVERYTHING as one event, doesn't seem to be breaking at the

Re: Why won't Splunk parse my multi-line event properly?

FrankVl — Mon, 09 Jul 2018 12:42:46 GMT

Your sample file contains a space before the <GPO... tag. Is that also there in the actual data? If so, you need to add change the line breaker to: ([\r\n]+\s+)\<GPO\sxmlns:xsd

Re: Why won't Splunk parse my multi-line event properly?

smcdonald20 — Mon, 09 Jul 2018 14:17:05 GMT

Your original change worked, i accidentally copied the "=" from the Line Breaker!
Thank you very much!