topic Re: XML tag extraction in Getting Data In

XML tag extraction

OMohi — Tue, 29 Sep 2020 06:59:57 GMT

I have a datasource that reads in events in XML format. Could someone please help me build a props.conf that will extract all fields and show the events in treeview. Sample event below:

Fri Aug 07 13:42:37 EDT 2015 name="QUEUE_msg_received" event_id="ID:414d51204d514942513032202020202055bdd7d620016441" msg_dest="QA.EA.ELOG.BUSINESSEVENT1" msg_body="<?xml version="1.0" encoding="UTF-8"?><v1:BusinessEventRequest xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:v1="http://schemas.humana.com/Infrastructure/Utility/Logging/BusinessEventRequest/V1.1"><v1:BusinessEvent><v1:BusinessEventMetaData><v1:BusinessEventTypeCode>BUSINESS_EVENT</v1:BusinessEventTypeCode><v1:BusinessEventDateTime>2015-08-07T01:43:47Z</v1:BusinessEventDateTime></v1:BusinessEventMetaData><v1:SourceApplicationInformation><v1:EAPMId>66666</v1:EAPMId><v1:HostMachineName>MQIBQ01</v1:HostMachineName><v1:HostEnvironmentName>QA</v1:HostEnvironmentName><v1:AppEventCorrelationId/><v1:Component><v1:ComponentId/><v1:ComponentName/></v1:Component></v1:SourceApplicationInformation><v1:BusinessProcessInformation><v1:ProcessName/><v1:EventModelXSDPath/><EventInformation><mstns:BAMEvent xmlns:mstns="http://enrollmentservices.humana.com/Schema/BAMSchema/v1.0"><mstns:EventSource>FileIntake</mstns:EventSource><mstns:Activity>FileIntakeActivity</mstns:Activity><mstns:EventTransactionId>40efe7da-4ef2-46b6-bea6-911a74db898e</mstns:EventTransactionId><mstns:EventCorrelationID>354805729</mstns:EventCorrelationID><mstns:Milestone><mstns:MilestoneEvent>File upload requested</mstns:MilestoneEvent><mstns:MilestoneState>Begin</mstns:MilestoneState><mstns:DataElements><mstns:FileName/><mstns:FileSize>9008</mstns:FileSize><mstns:AdditionalInfo>File upload requested</mstns:AdditionalInfo></mstns:DataElements></mstns:Milestone></mstns:BAMEvent></EventInformation></v1:BusinessProcessInformation></v1:BusinessEvent></v1:BusinessEventRequest>"

Re: XML tag extraction

sduff_splunk — Fri, 14 Aug 2015 01:10:59 GMT

In your props.conf, you should be able to use KV_MODE = xml to extract xml data

You could use the spath command in search to extract fields at search time
http://docs.splunk.com/Documentation/Splunk/6.2.4/SearchReference/spath

Re: XML tag extraction

OMohi — Fri, 14 Aug 2015 19:54:24 GMT

The problem is that I tried using KV_MODE = xml but the data contains some non xml fields hence the extraction doesn't work. I found a solution, that is defining in props.conf:

[sourcetype]
Report-xmlkv = xmlkv -alternative

In transforms.conf

[xmlkv-alternative]
REGEX = <([^\s>])[^>]>([^<]*)<\/\1>
FORMAT = $1::$2

This works and I was able to successfully extract all the XML tags as a field.

We can also | xmlkv for search time extraction but the client wanted the business users to understand the data in simplistic fashion.

Re: XML tag extraction

ejenson — Thu, 17 Sep 2015 14:39:46 GMT

This was very helpful for my situation where there is a mix of xml and non xml.
I had to tweak my regex in transforms.