Dashboards & Visualizations

Parse log and extract fields from xml

marcosrios
Explorer

Hi, I am trying to parse a log file through a regex and then parse the xml that is contained in a field.

This is the sample xml:

2016-03-04 00:02:05,546 DEBUG [net.ussouth.aps.shared.util.SysLogUtil] <?xml version="1.0" encoding="UTF-8"?><TxnReq xmlns="http://www.us-south.net/namespace/Transaction"><ReqCat>fastcard</ReqCat><ReqAction>Act</ReqAction><ReqRoute>dbservice</ReqRoute><TraceUID>8a848b128d531fe68940153400111f417ae</TraceUID><MerchDate>20160304</MerchDate><MerchTime>0002026</MerchTime><Product><Track1>;029874789000000000=</Track1><Track2>098722789000000000=</Track2><ExtSerNum>0987789000000000</ExtSerNum><UPC>07675004668</UPC><Denom>840</Denom><Amount>5.0</Amount></Product><Origin><MerchID>Toys R Us</MerchID><StoreID>0511927      TOYS</StoreID><TermID>051927330</TermID><TermType>FCMS</TermType><SrcRefNum>087591</SrcRefNum></Origin></TxnReq>

This is my query:

* | rex "(?<timestamp>(\d{4}-\d{2}-\d{2} \d{2}\:\d{2}\:\d{2}\,\d{3})) (?<loglevel>DEBUG|INFO|WARN|ERROR) (?<origin>\[.*\]) (?s)(?<content>.*)" | table timestamp, loglevel, origin, content

As you can see, I am setting the xml inside content field. Now, I want to parse "content" field and extract all the fields from it. I have tried with spath and xmlkv commands but cannot find the way to apply them as I am not getting the expected result. Could it be because of the first part of the xml <?xml version="1.0" encoding="UTF-8"?> ?

If you have a better way to parse the log and xml please let me know.
Regards!

1 Solution

marcosrios
Explorer

My bad. This is working.

source="txn.log" host="none" sourcetype= "txn log"| rex "(?<timestamp>(\d{4}-\d{2}-\d{2} \d{2}\:\d{2}\:\d{2}\,\d{3})) (?<loglevel>DEBUG|INFO|WARN|ERROR) (?<origin>\[.*\]) (?s)(?<content>.*)" | xmlkv

View solution in original post

0 Karma

marcosrios
Explorer

My bad. This is working.

source="txn.log" host="none" sourcetype= "txn log"| rex "(?<timestamp>(\d{4}-\d{2}-\d{2} \d{2}\:\d{2}\:\d{2}\,\d{3})) (?<loglevel>DEBUG|INFO|WARN|ERROR) (?<origin>\[.*\]) (?s)(?<content>.*)" | xmlkv
0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...