Dashboards & Visualizations

Parse log and extract fields from xml

marcosrios
Explorer

Hi, I am trying to parse a log file through a regex and then parse the xml that is contained in a field.

This is the sample xml:

2016-03-04 00:02:05,546 DEBUG [net.ussouth.aps.shared.util.SysLogUtil] <?xml version="1.0" encoding="UTF-8"?><TxnReq xmlns="http://www.us-south.net/namespace/Transaction"><ReqCat>fastcard</ReqCat><ReqAction>Act</ReqAction><ReqRoute>dbservice</ReqRoute><TraceUID>8a848b128d531fe68940153400111f417ae</TraceUID><MerchDate>20160304</MerchDate><MerchTime>0002026</MerchTime><Product><Track1>;029874789000000000=</Track1><Track2>098722789000000000=</Track2><ExtSerNum>0987789000000000</ExtSerNum><UPC>07675004668</UPC><Denom>840</Denom><Amount>5.0</Amount></Product><Origin><MerchID>Toys R Us</MerchID><StoreID>0511927      TOYS</StoreID><TermID>051927330</TermID><TermType>FCMS</TermType><SrcRefNum>087591</SrcRefNum></Origin></TxnReq>

This is my query:

* | rex "(?<timestamp>(\d{4}-\d{2}-\d{2} \d{2}\:\d{2}\:\d{2}\,\d{3})) (?<loglevel>DEBUG|INFO|WARN|ERROR) (?<origin>\[.*\]) (?s)(?<content>.*)" | table timestamp, loglevel, origin, content

As you can see, I am setting the xml inside content field. Now, I want to parse "content" field and extract all the fields from it. I have tried with spath and xmlkv commands but cannot find the way to apply them as I am not getting the expected result. Could it be because of the first part of the xml <?xml version="1.0" encoding="UTF-8"?> ?

If you have a better way to parse the log and xml please let me know.
Regards!

1 Solution

marcosrios
Explorer

My bad. This is working.

source="txn.log" host="none" sourcetype= "txn log"| rex "(?<timestamp>(\d{4}-\d{2}-\d{2} \d{2}\:\d{2}\:\d{2}\,\d{3})) (?<loglevel>DEBUG|INFO|WARN|ERROR) (?<origin>\[.*\]) (?s)(?<content>.*)" | xmlkv

View solution in original post

0 Karma

marcosrios
Explorer

My bad. This is working.

source="txn.log" host="none" sourcetype= "txn log"| rex "(?<timestamp>(\d{4}-\d{2}-\d{2} \d{2}\:\d{2}\:\d{2}\,\d{3})) (?<loglevel>DEBUG|INFO|WARN|ERROR) (?<origin>\[.*\]) (?s)(?<content>.*)" | xmlkv
0 Karma
Get Updates on the Splunk Community!

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...