Join the Conversation
- Find Answers
- :
- Splunk Administration
- :
- Getting Data In
- :
- XML tag extraction
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have a datasource that reads in events in XML format. Could someone please help me build a props.conf that will extract all fields and show the events in treeview. Sample event below:
Fri Aug 07 13:42:37 EDT 2015 name="QUEUE_msg_received" event_id="ID:414d51204d514942513032202020202055bdd7d620016441" msg_dest="QA.EA.ELOG.BUSINESSEVENT1" msg_body="<?xml version="1.0" encoding="UTF-8"?><v1:BusinessEventRequest xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:v1="http://schemas.humana.com/Infrastructure/Utility/Logging/BusinessEventRequest/V1.1"><v1:Busine... xmlns:mstns="http://enrollmentservices.humana.com/Schema/BAMSchema/v1.0"><mstns:EventSource>FileIntake&l... upload requested</mstns:MilestoneEvent><mstns:MilestoneState>Begin</mstns:MilestoneState><mstns:DataElements><mstns:FileName/><mstns:FileSize>9008</mstns:FileSize><mstns:AdditionalInfo>File upload requested</mstns:AdditionalInfo></mstns:DataElements></mstns:Milestone></mstns:BAMEvent></EventInformation></v1:BusinessProcessInformation></v1:BusinessEvent></v1:BusinessEventRequest>"
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The problem is that I tried using KV_MODE = xml but the data contains some non xml fields hence the extraction doesn't work. I found a solution, that is defining in props.conf:
[sourcetype]
Report-xmlkv = xmlkv -alternative
In transforms.conf
[xmlkv-alternative]
REGEX = <([^\s>])[^>]>([^<]*)<\/\1>
FORMAT = $1::$2
This works and I was able to successfully extract all the XML tags as a field.
We can also | xmlkv for search time extraction but the client wanted the business users to understand the data in simplistic fashion.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This was very helpful for my situation where there is a mix of xml and non xml.
I had to tweak my regex in transforms.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The problem is that I tried using KV_MODE = xml but the data contains some non xml fields hence the extraction doesn't work. I found a solution, that is defining in props.conf:
[sourcetype]
Report-xmlkv = xmlkv -alternative
In transforms.conf
[xmlkv-alternative]
REGEX = <([^\s>])[^>]>([^<]*)<\/\1>
FORMAT = $1::$2
This works and I was able to successfully extract all the XML tags as a field.
We can also | xmlkv for search time extraction but the client wanted the business users to understand the data in simplistic fashion.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
In your props.conf, you should be able to use KV_MODE = xml to extract xml data
You could use the spath command in search to extract fields at search time
http://docs.splunk.com/Documentation/Splunk/6.2.4/SearchReference/spath
Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!
Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.
Manage Edge Processor pipelines end to end with APIs
Splunk Brings Proof of Control to Digital Asset Transfers
Manage Ingest Processor pipelines end to end with APIs
