Splunk Search

How to use xmlkv in field extraction to identify values in XML?

3618475
Engager

I have a huge XML file with many tiers. I use this command to limit the number of events for the XML data that I want to extract, then I use xmlkv to extract the XML fields. The fields that I need are extracted but not all of the data is pulled.

This is the command:

index=83261 source="service.log" sourcetype="dispatchapp" "RULE" "createMessage MsgSource" | xmlkv

This is a partial example of the XML file:

PURCHASEDLINEHAUL DISPATCH    2020-05-21T17:22:55.000Z 
  <ns2:numberCode>923</ns2:numberCode>
  <ns2:numberType>2</ns2:numberType>    </origin>    <destination>
  <ns2:numberCode>72</ns2:numberCode>
  <ns2:numberType>2</ns2:numberType>    </destination>
<purchasedCost>
  <purchasedCostTripSegment>
     <purchCostReference>1587040</purchCostReference>
     <carrier>FXTR</carrier>
     <vendorType>DRAY</vendorType>
     <carrierTrailerType>PZ1</carrierTrailerType>
     <origin>
        <ns2:numberCode>923</ns2:numberCode>  

     </origin>
     <destination>
        <ns2:numberCode>4022</ns2:numberCode> 

     </destination>      
  </purchasedCostTripSegment>
  <purchasedCostTripSegment>
     <purchCostReference>1587040</purchCostReference>
     <carrier>BNSF</carrier>
     <vendorType>RAIL</vendorType>
     <carrierTrailerType>PZ1</carrierTrailerType>
     <origin>
        <ns2:numberCode>4022</ns2:numberCode> 

     </origin>
     <destination>
        <ns2:numberCode>4040</ns2:numberCode> 

     </destination>
  </purchasedCostTripSegment>
  <purchasedCostTripSegment>
     <purchCostReference>1587040</purchCostReference>
     <carrier>NS</carrier>
     <vendorType>RAIL</vendorType>
     <carrierTrailerType>PZ1</carrierTrailerType>
     <origin>
        <ns2:numberCode>4061</ns2:numberCode>
     </origin>
     <destination>
        <ns2:numberCode>4040</ns2:numberCode>
     </destination>     
  </purchasedCostTripSegment>    </purchasedCost>

This image shows that the ns:numberCode is extracted buy only 3 but there are 5 instances in the view above. But the entire xml there are many more.

alt text

How can I get the rest of the instances in the field extraction, and how can I identify the path where these values are coming from in the XML?

0 Karma

to4kawa
Ultra Champion

sample:

| makeresults
| eval _raw=" PURCHASEDLINEHAUL DISPATCH    2020-05-21T17:22:55.000Z 
   <ns2:numberCode>923</ns2:numberCode>
   <ns2:numberType>2</ns2:numberType>    </origin>    <destination>
   <ns2:numberCode>72</ns2:numberCode>
   <ns2:numberType>2</ns2:numberType>    </destination>
 <purchasedCost>
   <purchasedCostTripSegment>
      <purchCostReference>1587040</purchCostReference>
      <carrier>FXTR</carrier>
      <vendorType>DRAY</vendorType>
      <carrierTrailerType>PZ1</carrierTrailerType>
      <origin>
         <ns2:numberCode>923</ns2:numberCode>  
      </origin>
      <destination>
         <ns2:numberCode>4022</ns2:numberCode> 
      </destination>      
   </purchasedCostTripSegment>
   <purchasedCostTripSegment>
      <purchCostReference>1587040</purchCostReference>
      <carrier>BNSF</carrier>
      <vendorType>RAIL</vendorType>
      <carrierTrailerType>PZ1</carrierTrailerType>
      <origin>
         <ns2:numberCode>4022</ns2:numberCode> 
      </origin>
      <destination>
         <ns2:numberCode>4040</ns2:numberCode> 
      </destination>
   </purchasedCostTripSegment>
   <purchasedCostTripSegment>
      <purchCostReference>1587040</purchCostReference>
      <carrier>NS</carrier>
      <vendorType>RAIL</vendorType>
      <carrierTrailerType>PZ1</carrierTrailerType>
      <origin>
         <ns2:numberCode>4061</ns2:numberCode>
      </origin>
      <destination>
         <ns2:numberCode>4040</ns2:numberCode>
      </destination>     
   </purchasedCostTripSegment>    </purchasedCost>"
| multikv noheader=t 
| fields _raw
| xmlkv
| rex "(?<time>\d\S+Z)"
| stats list(*) as *

You have to think about the form of the output.

0 Karma
Get Updates on the Splunk Community!

Splunk Mobile: Your Brand-New Home Screen

Meet Your New Mobile Hub  Hello Splunk Community!  Staying connected to your data—no matter where you are—is ...

Introducing Value Insights (Beta): Understand the Business Impact your organization ...

Real progress on your strategic priorities starts with knowing the business outcomes your teams are delivering ...

Enterprise Security (ES) Essentials 8.3 is Now GA — Smarter Detections, Faster ...

As of today, Enterprise Security (ES) Essentials 8.3 is now generally available, helping SOC teams simplify ...