Dashboards & Visualizations

Parse log and extract fields from xml

marcosrios
Explorer

Hi, I am trying to parse a log file through a regex and then parse the xml that is contained in a field.

This is the sample xml:

2016-03-04 00:02:05,546 DEBUG [net.ussouth.aps.shared.util.SysLogUtil] <?xml version="1.0" encoding="UTF-8"?><TxnReq xmlns="http://www.us-south.net/namespace/Transaction"><ReqCat>fastcard</ReqCat><ReqAction>Act</ReqAction><ReqRoute>dbservice</ReqRoute><TraceUID>8a848b128d531fe68940153400111f417ae</TraceUID><MerchDate>20160304</MerchDate><MerchTime>0002026</MerchTime><Product><Track1>;029874789000000000=</Track1><Track2>098722789000000000=</Track2><ExtSerNum>0987789000000000</ExtSerNum><UPC>07675004668</UPC><Denom>840</Denom><Amount>5.0</Amount></Product><Origin><MerchID>Toys R Us</MerchID><StoreID>0511927      TOYS</StoreID><TermID>051927330</TermID><TermType>FCMS</TermType><SrcRefNum>087591</SrcRefNum></Origin></TxnReq>

This is my query:

* | rex "(?<timestamp>(\d{4}-\d{2}-\d{2} \d{2}\:\d{2}\:\d{2}\,\d{3})) (?<loglevel>DEBUG|INFO|WARN|ERROR) (?<origin>\[.*\]) (?s)(?<content>.*)" | table timestamp, loglevel, origin, content

As you can see, I am setting the xml inside content field. Now, I want to parse "content" field and extract all the fields from it. I have tried with spath and xmlkv commands but cannot find the way to apply them as I am not getting the expected result. Could it be because of the first part of the xml <?xml version="1.0" encoding="UTF-8"?> ?

If you have a better way to parse the log and xml please let me know.
Regards!

1 Solution

marcosrios
Explorer

My bad. This is working.

source="txn.log" host="none" sourcetype= "txn log"| rex "(?<timestamp>(\d{4}-\d{2}-\d{2} \d{2}\:\d{2}\:\d{2}\,\d{3})) (?<loglevel>DEBUG|INFO|WARN|ERROR) (?<origin>\[.*\]) (?s)(?<content>.*)" | xmlkv

View solution in original post

0 Karma

marcosrios
Explorer

My bad. This is working.

source="txn.log" host="none" sourcetype= "txn log"| rex "(?<timestamp>(\d{4}-\d{2}-\d{2} \d{2}\:\d{2}\:\d{2}\,\d{3})) (?<loglevel>DEBUG|INFO|WARN|ERROR) (?<origin>\[.*\]) (?s)(?<content>.*)" | xmlkv
0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...