Getting Data In

Is it possible to ingest XML?

nick405060
Motivator

It is 2019 and there is still not a comprehensive Splunk Answer or Documentation on how to ingest XML.

Can someone explain to me how to configure props to ingest

<?xml version="1.0" encoding="utf-8"?>
  <ArrayOfUser xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <User>
      <Id>removed</Id>
      <Uuid>removed</Uuid>
      ... many more attributes at this same level ...
    <User>
    <User>
      <Id>removed</Id>
      <Uuid>removed</Uuid>
      ... many more attributes at this same level ...
    <User>
0 Karma

woodcock
Esteemed Legend

Just bring it in and set KV_MODE = xml in props.conf for your sourcetype on your Search Head(s).

0 Karma

smoir_splunk
Splunk Employee
Splunk Employee

This is not going to be props for your specific case (assuming you want each one of those users to be a separate event) but a similar example for props and transforms that I have to extract tracks from an itunes library XML file.
They're also visible here:
https://github.com/smoreface/music_app_for_splunk/blob/master/default/transforms.conf
https://github.com/smoreface/music_app_for_splunk/blob/master/default/props.conf

transforms stanza:

[itunes_xml]
CLEAN_KEYS = true
FORMAT = $1::$2
REGEX = <key>([^<]+)</key><[^>]+>([^<]+)</

props stanza:
[itunes_xml]
FIELDALIAS-iTunes_xml_Normie = Album AS album Artist AS artist Date_Added AS date_added Name AS track_name Play_Count AS play_count Play_Date AS last_played Play_Date_UTC AS last_played_utc Rating AS rating Release_Date AS release_date Size AS file_size Total_Time AS track_length Track_Number AS track_number
SEDCMD-xml&to& = s/&#38;/&/g

Example XML being parsed:

<dict>
            <key>Track ID</key><integer>10815</integer>
            <key>Size</key><integer>4338490</integer>
            <key>Total Time</key><integer>216816</integer>
            <key>Track Number</key><integer>1</integer>
            <key>Track Count</key><integer>10</integer>
            <key>Year</key><integer>2004</integer>
            <key>Date Modified</key><date>2007-01-20T22:07:34Z</date>
            <key>Date Added</key><date>2008-07-27T03:52:43Z</date>
            <key>Bit Rate</key><integer>160</integer>
            <key>Sample Rate</key><integer>44100</integer>
            <key>Play Count</key><integer>1</integer>
            <key>Play Date</key><integer>3319660819</integer>
            <key>Play Date UTC</key><date>2009-03-12T07:00:19Z</date>
            <key>Skip Count</key><integer>1</integer>
            <key>Skip Date</key><date>2010-06-14T22:40:10Z</date>
            <key>Persistent ID</key><string>36990211F06BD125</string>
            <key>Track Type</key><string>File</string>
            <key>File Folder Count</key><integer>5</integer>
            <key>Library Folder Count</key><integer>1</integer>
            <key>Name</key><string>Cry</string>
            <key>Artist</key><string>Sirens</string>
            <key>Album</key><string>Tied To The Mast</string>
            <key>Genre</key><string>Pop</string>
            <key>Kind</key><string>MPEG audio file</string>
            <key>Location</key><string>file:///Users/user/Music/iTunes/iTunes%20Music/Music/Sirens/Tied%20To%20The%20Mast/01%20Cry.mp3</string>
        </dict>
        <key>10817</key>
        <dict>
            <key>Track ID</key><integer>10817</integer>
            <key>Size</key><integer>4082943</integer>
            <key>Total Time</key><integer>254093</integer>
            <key>Track Number</key><integer>1</integer>
            <key>Track Count</key><integer>2</integer>
            <key>Date Modified</key><date>2008-01-15T02:13:52Z</date>
            <key>Date Added</key><date>2008-07-27T03:52:43Z</date>
            <key>Bit Rate</key><integer>128</integer>
            <key>Sample Rate</key><integer>44100</integer>
            <key>Play Count</key><integer>19</integer>
            <key>Play Date</key><integer>3441386101</integer>
            <key>Play Date UTC</key><date>2013-01-19T04:35:01Z</date>
            <key>Skip Count</key><integer>1</integer>
            <key>Skip Date</key><date>2009-02-10T22:07:13Z</date>
            <key>Rating</key><integer>40</integer>
            <key>Album Rating</key><integer>20</integer>
            <key>Album Rating Computed</key><true/>
            <key>Persistent ID</key><string>36990211F06BD130</string>
            <key>Track Type</key><string>File</string>
            <key>File Folder Count</key><integer>5</integer>
            <key>Library Folder Count</key><integer>1</integer>
            <key>Name</key><string>Gone</string>
            <key>Artist</key><string>Straight No Chaser</string>
            <key>Kind</key><string>AAC audio file</string>
            <key>Location</key><string>file:///Users/user/Music/iTunes/iTunes%20Music/Music/Straight%20No%20Chaser/Unknown%20Album/01%20Gone.m4a</string>
        </dict>

Hope this helps!

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...