I have a huge XML file with many tiers. I use this command to limit the number of events for the XML data that I want to extract, then I use xmlkv to extract the XML fields. The fields that I need are extracted but not all of the data is pulled.
This is the command:
index=83261 source="service.log" sourcetype="dispatchapp" "RULE" "createMessage MsgSource" | xmlkv
This is a partial example of the XML file:
PURCHASEDLINEHAUL DISPATCH 2020-05-21T17:22:55.000Z
<ns2:numberCode>923</ns2:numberCode>
<ns2:numberType>2</ns2:numberType> </origin> <destination>
<ns2:numberCode>72</ns2:numberCode>
<ns2:numberType>2</ns2:numberType> </destination>
<purchasedCost>
<purchasedCostTripSegment>
<purchCostReference>1587040</purchCostReference>
<carrier>FXTR</carrier>
<vendorType>DRAY</vendorType>
<carrierTrailerType>PZ1</carrierTrailerType>
<origin>
<ns2:numberCode>923</ns2:numberCode>
</origin>
<destination>
<ns2:numberCode>4022</ns2:numberCode>
</destination>
</purchasedCostTripSegment>
<purchasedCostTripSegment>
<purchCostReference>1587040</purchCostReference>
<carrier>BNSF</carrier>
<vendorType>RAIL</vendorType>
<carrierTrailerType>PZ1</carrierTrailerType>
<origin>
<ns2:numberCode>4022</ns2:numberCode>
</origin>
<destination>
<ns2:numberCode>4040</ns2:numberCode>
</destination>
</purchasedCostTripSegment>
<purchasedCostTripSegment>
<purchCostReference>1587040</purchCostReference>
<carrier>NS</carrier>
<vendorType>RAIL</vendorType>
<carrierTrailerType>PZ1</carrierTrailerType>
<origin>
<ns2:numberCode>4061</ns2:numberCode>
</origin>
<destination>
<ns2:numberCode>4040</ns2:numberCode>
</destination>
</purchasedCostTripSegment> </purchasedCost>
This image shows that the ns:numberCode
is extracted buy only 3 but there are 5 instances in the view above. But the entire xml there are many more.
How can I get the rest of the instances in the field extraction, and how can I identify the path where these values are coming from in the XML?
sample:
| makeresults
| eval _raw=" PURCHASEDLINEHAUL DISPATCH 2020-05-21T17:22:55.000Z
<ns2:numberCode>923</ns2:numberCode>
<ns2:numberType>2</ns2:numberType> </origin> <destination>
<ns2:numberCode>72</ns2:numberCode>
<ns2:numberType>2</ns2:numberType> </destination>
<purchasedCost>
<purchasedCostTripSegment>
<purchCostReference>1587040</purchCostReference>
<carrier>FXTR</carrier>
<vendorType>DRAY</vendorType>
<carrierTrailerType>PZ1</carrierTrailerType>
<origin>
<ns2:numberCode>923</ns2:numberCode>
</origin>
<destination>
<ns2:numberCode>4022</ns2:numberCode>
</destination>
</purchasedCostTripSegment>
<purchasedCostTripSegment>
<purchCostReference>1587040</purchCostReference>
<carrier>BNSF</carrier>
<vendorType>RAIL</vendorType>
<carrierTrailerType>PZ1</carrierTrailerType>
<origin>
<ns2:numberCode>4022</ns2:numberCode>
</origin>
<destination>
<ns2:numberCode>4040</ns2:numberCode>
</destination>
</purchasedCostTripSegment>
<purchasedCostTripSegment>
<purchCostReference>1587040</purchCostReference>
<carrier>NS</carrier>
<vendorType>RAIL</vendorType>
<carrierTrailerType>PZ1</carrierTrailerType>
<origin>
<ns2:numberCode>4061</ns2:numberCode>
</origin>
<destination>
<ns2:numberCode>4040</ns2:numberCode>
</destination>
</purchasedCostTripSegment> </purchasedCost>"
| multikv noheader=t
| fields _raw
| xmlkv
| rex "(?<time>\d\S+Z)"
| stats list(*) as *
You have to think about the form of the output.