Splunk Search

How to use xmlkv in field extraction to identify values in XML?

3618475
Engager

I have a huge XML file with many tiers. I use this command to limit the number of events for the XML data that I want to extract, then I use xmlkv to extract the XML fields. The fields that I need are extracted but not all of the data is pulled.

This is the command:

index=83261 source="service.log" sourcetype="dispatchapp" "RULE" "createMessage MsgSource" | xmlkv

This is a partial example of the XML file:

PURCHASEDLINEHAUL DISPATCH    2020-05-21T17:22:55.000Z 
  <ns2:numberCode>923</ns2:numberCode>
  <ns2:numberType>2</ns2:numberType>    </origin>    <destination>
  <ns2:numberCode>72</ns2:numberCode>
  <ns2:numberType>2</ns2:numberType>    </destination>
<purchasedCost>
  <purchasedCostTripSegment>
     <purchCostReference>1587040</purchCostReference>
     <carrier>FXTR</carrier>
     <vendorType>DRAY</vendorType>
     <carrierTrailerType>PZ1</carrierTrailerType>
     <origin>
        <ns2:numberCode>923</ns2:numberCode>  

     </origin>
     <destination>
        <ns2:numberCode>4022</ns2:numberCode> 

     </destination>      
  </purchasedCostTripSegment>
  <purchasedCostTripSegment>
     <purchCostReference>1587040</purchCostReference>
     <carrier>BNSF</carrier>
     <vendorType>RAIL</vendorType>
     <carrierTrailerType>PZ1</carrierTrailerType>
     <origin>
        <ns2:numberCode>4022</ns2:numberCode> 

     </origin>
     <destination>
        <ns2:numberCode>4040</ns2:numberCode> 

     </destination>
  </purchasedCostTripSegment>
  <purchasedCostTripSegment>
     <purchCostReference>1587040</purchCostReference>
     <carrier>NS</carrier>
     <vendorType>RAIL</vendorType>
     <carrierTrailerType>PZ1</carrierTrailerType>
     <origin>
        <ns2:numberCode>4061</ns2:numberCode>
     </origin>
     <destination>
        <ns2:numberCode>4040</ns2:numberCode>
     </destination>     
  </purchasedCostTripSegment>    </purchasedCost>

This image shows that the ns:numberCode is extracted buy only 3 but there are 5 instances in the view above. But the entire xml there are many more.

alt text

How can I get the rest of the instances in the field extraction, and how can I identify the path where these values are coming from in the XML?

0 Karma

to4kawa
Ultra Champion

sample:

| makeresults
| eval _raw=" PURCHASEDLINEHAUL DISPATCH    2020-05-21T17:22:55.000Z 
   <ns2:numberCode>923</ns2:numberCode>
   <ns2:numberType>2</ns2:numberType>    </origin>    <destination>
   <ns2:numberCode>72</ns2:numberCode>
   <ns2:numberType>2</ns2:numberType>    </destination>
 <purchasedCost>
   <purchasedCostTripSegment>
      <purchCostReference>1587040</purchCostReference>
      <carrier>FXTR</carrier>
      <vendorType>DRAY</vendorType>
      <carrierTrailerType>PZ1</carrierTrailerType>
      <origin>
         <ns2:numberCode>923</ns2:numberCode>  
      </origin>
      <destination>
         <ns2:numberCode>4022</ns2:numberCode> 
      </destination>      
   </purchasedCostTripSegment>
   <purchasedCostTripSegment>
      <purchCostReference>1587040</purchCostReference>
      <carrier>BNSF</carrier>
      <vendorType>RAIL</vendorType>
      <carrierTrailerType>PZ1</carrierTrailerType>
      <origin>
         <ns2:numberCode>4022</ns2:numberCode> 
      </origin>
      <destination>
         <ns2:numberCode>4040</ns2:numberCode> 
      </destination>
   </purchasedCostTripSegment>
   <purchasedCostTripSegment>
      <purchCostReference>1587040</purchCostReference>
      <carrier>NS</carrier>
      <vendorType>RAIL</vendorType>
      <carrierTrailerType>PZ1</carrierTrailerType>
      <origin>
         <ns2:numberCode>4061</ns2:numberCode>
      </origin>
      <destination>
         <ns2:numberCode>4040</ns2:numberCode>
      </destination>     
   </purchasedCostTripSegment>    </purchasedCost>"
| multikv noheader=t 
| fields _raw
| xmlkv
| rex "(?<time>\d\S+Z)"
| stats list(*) as *

You have to think about the form of the output.

0 Karma
Get Updates on the Splunk Community!

ATTENTION!! We’re MOVING (not really)

Hey, all! In an effort to keep this Slack workspace secure and also to make our new members' experience easy, ...

Splunk Admins: Build a Smarter Stack with These Must-See .conf25 Sessions

  Whether you're running a complex Splunk deployment or just getting your bearings as a new admin, .conf25 ...

AppDynamics Summer Webinars

This summer, our mighty AppDynamics team is cooking up some delicious content on YouTube Live to satiate your ...