Getting Data In

Is it possible to ingest XML?

nick405060
Motivator

It is 2019 and there is still not a comprehensive Splunk Answer or Documentation on how to ingest XML.

Can someone explain to me how to configure props to ingest

<?xml version="1.0" encoding="utf-8"?>
  <ArrayOfUser xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <User>
      <Id>removed</Id>
      <Uuid>removed</Uuid>
      ... many more attributes at this same level ...
    <User>
    <User>
      <Id>removed</Id>
      <Uuid>removed</Uuid>
      ... many more attributes at this same level ...
    <User>
0 Karma

woodcock
Esteemed Legend

Just bring it in and set KV_MODE = xml in props.conf for your sourcetype on your Search Head(s).

0 Karma

smoir_splunk
Splunk Employee
Splunk Employee

This is not going to be props for your specific case (assuming you want each one of those users to be a separate event) but a similar example for props and transforms that I have to extract tracks from an itunes library XML file.
They're also visible here:
https://github.com/smoreface/music_app_for_splunk/blob/master/default/transforms.conf
https://github.com/smoreface/music_app_for_splunk/blob/master/default/props.conf

transforms stanza:

[itunes_xml]
CLEAN_KEYS = true
FORMAT = $1::$2
REGEX = <key>([^<]+)</key><[^>]+>([^<]+)</

props stanza:
[itunes_xml]
FIELDALIAS-iTunes_xml_Normie = Album AS album Artist AS artist Date_Added AS date_added Name AS track_name Play_Count AS play_count Play_Date AS last_played Play_Date_UTC AS last_played_utc Rating AS rating Release_Date AS release_date Size AS file_size Total_Time AS track_length Track_Number AS track_number
SEDCMD-xml&to& = s/&#38;/&/g

Example XML being parsed:

<dict>
            <key>Track ID</key><integer>10815</integer>
            <key>Size</key><integer>4338490</integer>
            <key>Total Time</key><integer>216816</integer>
            <key>Track Number</key><integer>1</integer>
            <key>Track Count</key><integer>10</integer>
            <key>Year</key><integer>2004</integer>
            <key>Date Modified</key><date>2007-01-20T22:07:34Z</date>
            <key>Date Added</key><date>2008-07-27T03:52:43Z</date>
            <key>Bit Rate</key><integer>160</integer>
            <key>Sample Rate</key><integer>44100</integer>
            <key>Play Count</key><integer>1</integer>
            <key>Play Date</key><integer>3319660819</integer>
            <key>Play Date UTC</key><date>2009-03-12T07:00:19Z</date>
            <key>Skip Count</key><integer>1</integer>
            <key>Skip Date</key><date>2010-06-14T22:40:10Z</date>
            <key>Persistent ID</key><string>36990211F06BD125</string>
            <key>Track Type</key><string>File</string>
            <key>File Folder Count</key><integer>5</integer>
            <key>Library Folder Count</key><integer>1</integer>
            <key>Name</key><string>Cry</string>
            <key>Artist</key><string>Sirens</string>
            <key>Album</key><string>Tied To The Mast</string>
            <key>Genre</key><string>Pop</string>
            <key>Kind</key><string>MPEG audio file</string>
            <key>Location</key><string>file:///Users/user/Music/iTunes/iTunes%20Music/Music/Sirens/Tied%20To%20The%20Mast/01%20Cry.mp3</string>
        </dict>
        <key>10817</key>
        <dict>
            <key>Track ID</key><integer>10817</integer>
            <key>Size</key><integer>4082943</integer>
            <key>Total Time</key><integer>254093</integer>
            <key>Track Number</key><integer>1</integer>
            <key>Track Count</key><integer>2</integer>
            <key>Date Modified</key><date>2008-01-15T02:13:52Z</date>
            <key>Date Added</key><date>2008-07-27T03:52:43Z</date>
            <key>Bit Rate</key><integer>128</integer>
            <key>Sample Rate</key><integer>44100</integer>
            <key>Play Count</key><integer>19</integer>
            <key>Play Date</key><integer>3441386101</integer>
            <key>Play Date UTC</key><date>2013-01-19T04:35:01Z</date>
            <key>Skip Count</key><integer>1</integer>
            <key>Skip Date</key><date>2009-02-10T22:07:13Z</date>
            <key>Rating</key><integer>40</integer>
            <key>Album Rating</key><integer>20</integer>
            <key>Album Rating Computed</key><true/>
            <key>Persistent ID</key><string>36990211F06BD130</string>
            <key>Track Type</key><string>File</string>
            <key>File Folder Count</key><integer>5</integer>
            <key>Library Folder Count</key><integer>1</integer>
            <key>Name</key><string>Gone</string>
            <key>Artist</key><string>Straight No Chaser</string>
            <key>Kind</key><string>AAC audio file</string>
            <key>Location</key><string>file:///Users/user/Music/iTunes/iTunes%20Music/Music/Straight%20No%20Chaser/Unknown%20Album/01%20Gone.m4a</string>
        </dict>

Hope this helps!

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Can’t Make It to Boston? Stream .conf25 and Learn with Haya Husain

Boston may be buzzing this September with Splunk University and .conf25, but you don’t have to pack a bag to ...

Splunk Lantern’s Guide to The Most Popular .conf25 Sessions

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Unlock What’s Next: The Splunk Cloud Platform at .conf25

In just a few days, Boston will be buzzing as the Splunk team and thousands of community members come together ...