Splunk Search

How to use xmlkv in field extraction to identify values in XML?

3618475
Engager

I have a huge XML file with many tiers. I use this command to limit the number of events for the XML data that I want to extract, then I use xmlkv to extract the XML fields. The fields that I need are extracted but not all of the data is pulled.

This is the command:

index=83261 source="service.log" sourcetype="dispatchapp" "RULE" "createMessage MsgSource" | xmlkv

This is a partial example of the XML file:

PURCHASEDLINEHAUL DISPATCH    2020-05-21T17:22:55.000Z 
  <ns2:numberCode>923</ns2:numberCode>
  <ns2:numberType>2</ns2:numberType>    </origin>    <destination>
  <ns2:numberCode>72</ns2:numberCode>
  <ns2:numberType>2</ns2:numberType>    </destination>
<purchasedCost>
  <purchasedCostTripSegment>
     <purchCostReference>1587040</purchCostReference>
     <carrier>FXTR</carrier>
     <vendorType>DRAY</vendorType>
     <carrierTrailerType>PZ1</carrierTrailerType>
     <origin>
        <ns2:numberCode>923</ns2:numberCode>  

     </origin>
     <destination>
        <ns2:numberCode>4022</ns2:numberCode> 

     </destination>      
  </purchasedCostTripSegment>
  <purchasedCostTripSegment>
     <purchCostReference>1587040</purchCostReference>
     <carrier>BNSF</carrier>
     <vendorType>RAIL</vendorType>
     <carrierTrailerType>PZ1</carrierTrailerType>
     <origin>
        <ns2:numberCode>4022</ns2:numberCode> 

     </origin>
     <destination>
        <ns2:numberCode>4040</ns2:numberCode> 

     </destination>
  </purchasedCostTripSegment>
  <purchasedCostTripSegment>
     <purchCostReference>1587040</purchCostReference>
     <carrier>NS</carrier>
     <vendorType>RAIL</vendorType>
     <carrierTrailerType>PZ1</carrierTrailerType>
     <origin>
        <ns2:numberCode>4061</ns2:numberCode>
     </origin>
     <destination>
        <ns2:numberCode>4040</ns2:numberCode>
     </destination>     
  </purchasedCostTripSegment>    </purchasedCost>

This image shows that the ns:numberCode is extracted buy only 3 but there are 5 instances in the view above. But the entire xml there are many more.

alt text

How can I get the rest of the instances in the field extraction, and how can I identify the path where these values are coming from in the XML?

0 Karma

to4kawa
Ultra Champion

sample:

| makeresults
| eval _raw=" PURCHASEDLINEHAUL DISPATCH    2020-05-21T17:22:55.000Z 
   <ns2:numberCode>923</ns2:numberCode>
   <ns2:numberType>2</ns2:numberType>    </origin>    <destination>
   <ns2:numberCode>72</ns2:numberCode>
   <ns2:numberType>2</ns2:numberType>    </destination>
 <purchasedCost>
   <purchasedCostTripSegment>
      <purchCostReference>1587040</purchCostReference>
      <carrier>FXTR</carrier>
      <vendorType>DRAY</vendorType>
      <carrierTrailerType>PZ1</carrierTrailerType>
      <origin>
         <ns2:numberCode>923</ns2:numberCode>  
      </origin>
      <destination>
         <ns2:numberCode>4022</ns2:numberCode> 
      </destination>      
   </purchasedCostTripSegment>
   <purchasedCostTripSegment>
      <purchCostReference>1587040</purchCostReference>
      <carrier>BNSF</carrier>
      <vendorType>RAIL</vendorType>
      <carrierTrailerType>PZ1</carrierTrailerType>
      <origin>
         <ns2:numberCode>4022</ns2:numberCode> 
      </origin>
      <destination>
         <ns2:numberCode>4040</ns2:numberCode> 
      </destination>
   </purchasedCostTripSegment>
   <purchasedCostTripSegment>
      <purchCostReference>1587040</purchCostReference>
      <carrier>NS</carrier>
      <vendorType>RAIL</vendorType>
      <carrierTrailerType>PZ1</carrierTrailerType>
      <origin>
         <ns2:numberCode>4061</ns2:numberCode>
      </origin>
      <destination>
         <ns2:numberCode>4040</ns2:numberCode>
      </destination>     
   </purchasedCostTripSegment>    </purchasedCost>"
| multikv noheader=t 
| fields _raw
| xmlkv
| rex "(?<time>\d\S+Z)"
| stats list(*) as *

You have to think about the form of the output.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...