Getting Data In

Is it possible to ingest XML?

nick405060
Motivator

It is 2019 and there is still not a comprehensive Splunk Answer or Documentation on how to ingest XML.

Can someone explain to me how to configure props to ingest

<?xml version="1.0" encoding="utf-8"?>
  <ArrayOfUser xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <User>
      <Id>removed</Id>
      <Uuid>removed</Uuid>
      ... many more attributes at this same level ...
    <User>
    <User>
      <Id>removed</Id>
      <Uuid>removed</Uuid>
      ... many more attributes at this same level ...
    <User>
0 Karma

woodcock
Esteemed Legend

Just bring it in and set KV_MODE = xml in props.conf for your sourcetype on your Search Head(s).

0 Karma

smoir_splunk
Splunk Employee
Splunk Employee

This is not going to be props for your specific case (assuming you want each one of those users to be a separate event) but a similar example for props and transforms that I have to extract tracks from an itunes library XML file.
They're also visible here:
https://github.com/smoreface/music_app_for_splunk/blob/master/default/transforms.conf
https://github.com/smoreface/music_app_for_splunk/blob/master/default/props.conf

transforms stanza:

[itunes_xml]
CLEAN_KEYS = true
FORMAT = $1::$2
REGEX = <key>([^<]+)</key><[^>]+>([^<]+)</

props stanza:
[itunes_xml]
FIELDALIAS-iTunes_xml_Normie = Album AS album Artist AS artist Date_Added AS date_added Name AS track_name Play_Count AS play_count Play_Date AS last_played Play_Date_UTC AS last_played_utc Rating AS rating Release_Date AS release_date Size AS file_size Total_Time AS track_length Track_Number AS track_number
SEDCMD-xml&to& = s/&#38;/&/g

Example XML being parsed:

<dict>
            <key>Track ID</key><integer>10815</integer>
            <key>Size</key><integer>4338490</integer>
            <key>Total Time</key><integer>216816</integer>
            <key>Track Number</key><integer>1</integer>
            <key>Track Count</key><integer>10</integer>
            <key>Year</key><integer>2004</integer>
            <key>Date Modified</key><date>2007-01-20T22:07:34Z</date>
            <key>Date Added</key><date>2008-07-27T03:52:43Z</date>
            <key>Bit Rate</key><integer>160</integer>
            <key>Sample Rate</key><integer>44100</integer>
            <key>Play Count</key><integer>1</integer>
            <key>Play Date</key><integer>3319660819</integer>
            <key>Play Date UTC</key><date>2009-03-12T07:00:19Z</date>
            <key>Skip Count</key><integer>1</integer>
            <key>Skip Date</key><date>2010-06-14T22:40:10Z</date>
            <key>Persistent ID</key><string>36990211F06BD125</string>
            <key>Track Type</key><string>File</string>
            <key>File Folder Count</key><integer>5</integer>
            <key>Library Folder Count</key><integer>1</integer>
            <key>Name</key><string>Cry</string>
            <key>Artist</key><string>Sirens</string>
            <key>Album</key><string>Tied To The Mast</string>
            <key>Genre</key><string>Pop</string>
            <key>Kind</key><string>MPEG audio file</string>
            <key>Location</key><string>file:///Users/user/Music/iTunes/iTunes%20Music/Music/Sirens/Tied%20To%20The%20Mast/01%20Cry.mp3</string>
        </dict>
        <key>10817</key>
        <dict>
            <key>Track ID</key><integer>10817</integer>
            <key>Size</key><integer>4082943</integer>
            <key>Total Time</key><integer>254093</integer>
            <key>Track Number</key><integer>1</integer>
            <key>Track Count</key><integer>2</integer>
            <key>Date Modified</key><date>2008-01-15T02:13:52Z</date>
            <key>Date Added</key><date>2008-07-27T03:52:43Z</date>
            <key>Bit Rate</key><integer>128</integer>
            <key>Sample Rate</key><integer>44100</integer>
            <key>Play Count</key><integer>19</integer>
            <key>Play Date</key><integer>3441386101</integer>
            <key>Play Date UTC</key><date>2013-01-19T04:35:01Z</date>
            <key>Skip Count</key><integer>1</integer>
            <key>Skip Date</key><date>2009-02-10T22:07:13Z</date>
            <key>Rating</key><integer>40</integer>
            <key>Album Rating</key><integer>20</integer>
            <key>Album Rating Computed</key><true/>
            <key>Persistent ID</key><string>36990211F06BD130</string>
            <key>Track Type</key><string>File</string>
            <key>File Folder Count</key><integer>5</integer>
            <key>Library Folder Count</key><integer>1</integer>
            <key>Name</key><string>Gone</string>
            <key>Artist</key><string>Straight No Chaser</string>
            <key>Kind</key><string>AAC audio file</string>
            <key>Location</key><string>file:///Users/user/Music/iTunes/iTunes%20Music/Music/Straight%20No%20Chaser/Unknown%20Album/01%20Gone.m4a</string>
        </dict>

Hope this helps!

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...