topic Re: Extract multi valued XML field as key/value pairs in Dashboards & Visualizations

Extract multi valued XML field as key/value pairs

gustavomichels — Mon, 05 May 2014 15:25:51 GMT

Hello,

I have the following XML (the dots are removed parts):

<Exception timestamp="05/05/2014 14:25:53" ...>
<StackTrace>
    <Frame exceptionType=...>
        <Context>
            <Data name="Request_ApplicationPath">the_path</Data>
            <Data name="Request_Url_AbsoluteUri">the_url</Data>
            <Data name="ApplicationName">the_appname</Data>
        </Context>
    </Frame>
</StackTrace>

How can I get key/value fields as:

Request_ApplicationPath = the_path
Request_Url_AbsoluteUri = the_url
ApplicationName = the_appname

I'm trying to use spath but all I can get is either the key or the value, not the combination of them as fields.

Thank you,

Edit: I was looking for a solution without having to rely on regexes... I'm sorry, I should have mentioned it in the original post.

Re: Extract multi valued XML field as key/value pairs

martin_mueller — Mon, 05 May 2014 18:31:53 GMT

You can write a props.conf/transforms.conf extraction something like this:

props.conf

[your_sourcetype]
TRANSFORMS-data = extract_data

transforms.conf

[extract_data]
REGEX = Data\s+name="(?<_KEY_1>[^"])"\s*>(?<_VAL_1>[^<]+)

See http://docs.splunk.com/Documentation/Splunk/6.0.3/Admin/transformsconf for reference, search for _KEY_.
Note, this makes assumptions about your XML, for example that name is the only attribute of the Data element.

Re: Extract multi valued XML field as key/value pairs

gustavomichels — Mon, 05 May 2014 18:36:54 GMT

Thanks for your answer.

I was looking for a solution without having to rely on regexes... I'm sorry, I should have mentioned it in the original post.

Re: Extract multi valued XML field as key/value pairs

antlefebvre — Mon, 05 May 2014 19:20:54 GMT

In your props.conf do

[sourcetype]
EXTRACT-prsnl_id = (?i)<prsnl_id>(?P<prsnl_id>[^<]+)

Where prsnl_id is the xml tag you would be trying to create the key value pair.

Re: Extract multi valued XML field as key/value pairs

gustavomichels — Mon, 05 May 2014 19:33:09 GMT

Thanks for your answer.

I was looking for a solution without having to rely on regexes... I'm sorry, I should have mentioned it in the original post, but it won't let me edit as the captcha always fails.

Re: Extract multi valued XML field as key/value pairs

martin_mueller — Mon, 05 May 2014 19:39:40 GMT

I've added that to your question.

Getting field names from your event data without this approach of extracting the field name in transforms.conf isn't going to work. spath can extract the name into one field value and the value into another field value, but I don't see a simple way of getting that used as a field name.

Re: Extract multi valued XML field as key/value pairs

tamakg — Fri, 24 Aug 2018 18:55:40 GMT

Did you solve this issue? I have the same challenge right now...

Re: Extract multi valued XML field as key/value pairs

ddrillic — Sat, 25 Aug 2018 23:55:49 GMT

My suggestion is to clean a bit the xml document, to be something like -

 <Exception timestamp="05/05/2014 14:25:53" ...>

   <StackTrace>
      <Frame exceptionType=...>
           <Context>
               <Request_ApplicationPath>the_path</Request_ApplicationPath>
               <Request_Url_AbsoluteUri>the_url</Request_Url_AbsoluteUri>
               <ApplicationName>the_appname</ApplicationName>
           </Context>
      </Frame>
 </StackTrace>

By using the xml sourcetype, you can reach the xml elements via - | spath StackTrace.Frame.Context.Request_ApplicationPath.