Parsing oddly formatted XML (NetApp log)

RickH — Mon, 08 May 2017 11:16:22 GMT

Hello, I'm trying to parse a XML file generated by NetApp security auditing. I'm able to get it ingested, but it's just a string, so it's hard for the consumers to decode. The XML itself is confusingly formatted, I'm baffled on how to parse. I wouldn't mind extracting the key/values and dumping the raw message.

My props is:

   NO_BINARY_CHECK = true
    category = Custom
    description = netapp
    disabled = false
    pulldown_type = true
    BREAK_ONLY_BEFORE = ^<Event>
    KV_MODE = xml
    TIME_PREFIX = TimeCreated SystemTime\=

The data looks like this(except all run together - no line breaks):

    <Event> 
    <System>
    <Provider Name="Netapp-Security-Auditing"/>
    <EventID>4656</EventID>
    <EventName>Open Object</EventName>
    <Version>011.3</Version>
    <Source>CIFS</Source>
    <Level>0</Level>
    <Opcode>0</Opcode>
    <Keywords>0x8020000000000000</Keywords>
    <Result>Audit Success</Result>
    <TimeCreated SystemTime="2017-05-05T21:05:15.082531000Z"/>
    <Correlation/>
    <Channel>Security</Channel>
    <Computer>22b28c65-f331-11e4-a097-22a098654934/a1ee37f6-f8d5-11e4-a098-00a098654934</Computer>
    <Security/>
    </System>
    <EventData>
   ##lines like below have variations on how they show the key=values
    <Data Name="SubjectIP" IPVersion="4">10.22.33.235</Data>
    <Data Name="SubjectUnix" Uid="65534" Gid="65534" Local="false"></Data>
    <Data Name="SubjectUserSid">S-1-5-21-2112345321-1480738125-1508530778-84740</Data>
    <Data Name="SubjectUserIsLocal">false</Data>
    <Data Name="SubjectDomainName">theDomain</Data>
    <Data Name="SubjectUserName">Bob12345</Data>
    <Data Name="ObjectServer">Security</Data>
    <Data Name="ObjectType">File</Data>
    <Data Name="HandleID">00000000000424;00;00069642;b4221099</Data>
    <Data Name="ObjectName">(design);/Jobs/RR41_Video/RR41_PRINT/~mc_rr41~3.idlk</Data>
    <Data Name="AccessList">%%4423 %%1537 %%1541 </Data>
    <Data Name="AccessMask">11080</Data>
    <Data Name="DesiredAccess">Read Attributes; Delete; Synchronize; </Data>
    <Data Name="Attributes"></Data>
    </EventData>
    </Event>

Thanks!

Re: Parsing oddly formatted XML (NetApp log)

adonio — Mon, 08 May 2017 14:25:57 GMT

you can use the spath command after its been indexed
nice examples here in answers:
https://answers.splunk.com/answers/2141/xml-log-source-type.html
https://answers.splunk.com/answers/28619/indexing-xml-log-file-input.html
hope it helps

Re: Parsing oddly formatted XML (NetApp log)

RickH — Mon, 08 May 2017 20:22:53 GMT

Thanks, I'll try spath out next. I am able to get some meaningful data out with xmlkv & rex (below). xmlkv gets some of the fields and the rex grabs a few more.
My preference is to extract at index time so that I only store the fields I care about, but still researching - haven't figured that out yet.

index=netapp | xmlkv | search EventName="*"
|rex "\<Data Name\=\"ObjectName\"\>(?<file>.*?)\<\/Data\>"
| rex "\<Data Name\=\"OldPath\"\>(?<oldfile>.*?)\<\/Data\>" 
| rex "\<Data Name\=\"NewPath\"\>(?<newfile>.*?)\<\/Data\>" 
|table _time, EventName, file, oldfile, newfile

topic Parsing oddly formatted XML (NetApp log) in Dashboards & Visualizations

Parsing oddly formatted XML (NetApp log)

Re: Parsing oddly formatted XML (NetApp log)

Re: Parsing oddly formatted XML (NetApp log)