Splunk Search

How to use xmlkv in field extraction to identify values in XML?

3618475
Engager

I have a huge XML file with many tiers. I use this command to limit the number of events for the XML data that I want to extract, then I use xmlkv to extract the XML fields. The fields that I need are extracted but not all of the data is pulled.

This is the command:

index=83261 source="service.log" sourcetype="dispatchapp" "RULE" "createMessage MsgSource" | xmlkv

This is a partial example of the XML file:

PURCHASEDLINEHAUL DISPATCH    2020-05-21T17:22:55.000Z 
  <ns2:numberCode>923</ns2:numberCode>
  <ns2:numberType>2</ns2:numberType>    </origin>    <destination>
  <ns2:numberCode>72</ns2:numberCode>
  <ns2:numberType>2</ns2:numberType>    </destination>
<purchasedCost>
  <purchasedCostTripSegment>
     <purchCostReference>1587040</purchCostReference>
     <carrier>FXTR</carrier>
     <vendorType>DRAY</vendorType>
     <carrierTrailerType>PZ1</carrierTrailerType>
     <origin>
        <ns2:numberCode>923</ns2:numberCode>  

     </origin>
     <destination>
        <ns2:numberCode>4022</ns2:numberCode> 

     </destination>      
  </purchasedCostTripSegment>
  <purchasedCostTripSegment>
     <purchCostReference>1587040</purchCostReference>
     <carrier>BNSF</carrier>
     <vendorType>RAIL</vendorType>
     <carrierTrailerType>PZ1</carrierTrailerType>
     <origin>
        <ns2:numberCode>4022</ns2:numberCode> 

     </origin>
     <destination>
        <ns2:numberCode>4040</ns2:numberCode> 

     </destination>
  </purchasedCostTripSegment>
  <purchasedCostTripSegment>
     <purchCostReference>1587040</purchCostReference>
     <carrier>NS</carrier>
     <vendorType>RAIL</vendorType>
     <carrierTrailerType>PZ1</carrierTrailerType>
     <origin>
        <ns2:numberCode>4061</ns2:numberCode>
     </origin>
     <destination>
        <ns2:numberCode>4040</ns2:numberCode>
     </destination>     
  </purchasedCostTripSegment>    </purchasedCost>

This image shows that the ns:numberCode is extracted buy only 3 but there are 5 instances in the view above. But the entire xml there are many more.

alt text

How can I get the rest of the instances in the field extraction, and how can I identify the path where these values are coming from in the XML?

0 Karma

to4kawa
Ultra Champion

sample:

| makeresults
| eval _raw=" PURCHASEDLINEHAUL DISPATCH    2020-05-21T17:22:55.000Z 
   <ns2:numberCode>923</ns2:numberCode>
   <ns2:numberType>2</ns2:numberType>    </origin>    <destination>
   <ns2:numberCode>72</ns2:numberCode>
   <ns2:numberType>2</ns2:numberType>    </destination>
 <purchasedCost>
   <purchasedCostTripSegment>
      <purchCostReference>1587040</purchCostReference>
      <carrier>FXTR</carrier>
      <vendorType>DRAY</vendorType>
      <carrierTrailerType>PZ1</carrierTrailerType>
      <origin>
         <ns2:numberCode>923</ns2:numberCode>  
      </origin>
      <destination>
         <ns2:numberCode>4022</ns2:numberCode> 
      </destination>      
   </purchasedCostTripSegment>
   <purchasedCostTripSegment>
      <purchCostReference>1587040</purchCostReference>
      <carrier>BNSF</carrier>
      <vendorType>RAIL</vendorType>
      <carrierTrailerType>PZ1</carrierTrailerType>
      <origin>
         <ns2:numberCode>4022</ns2:numberCode> 
      </origin>
      <destination>
         <ns2:numberCode>4040</ns2:numberCode> 
      </destination>
   </purchasedCostTripSegment>
   <purchasedCostTripSegment>
      <purchCostReference>1587040</purchCostReference>
      <carrier>NS</carrier>
      <vendorType>RAIL</vendorType>
      <carrierTrailerType>PZ1</carrierTrailerType>
      <origin>
         <ns2:numberCode>4061</ns2:numberCode>
      </origin>
      <destination>
         <ns2:numberCode>4040</ns2:numberCode>
      </destination>     
   </purchasedCostTripSegment>    </purchasedCost>"
| multikv noheader=t 
| fields _raw
| xmlkv
| rex "(?<time>\d\S+Z)"
| stats list(*) as *

You have to think about the form of the output.

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...