Dashboards & Visualizations

Parse Windows RDGateway XML

ZimmermanC1
Explorer

Hey all,

I am really struggling to create a parser for a specific section of the Windows-TerminalServices-Gateway/Operational Event log.

Everything I have added to props & transforms appears to be correct, and I can get the sections I want to parse via SPL but whenever I add it to the sourcetype and refresh nothing changes.

inputs.conf on the Windows 2016 Server

[WinEventLog://Microsoft-Windows-TerminalServices-Gateway/Operational]
disabled = 0
start_from = oldest
current_only = 0
checkpointInterval = 5
renderXml = true
index = windows

[WinEventLog://Microsoft-Windows-TerminalServices-Gateway/Admin]
disabled = 0
start_from = oldest
current_only = 0
checkpointInterval = 5
renderXml = true
index = windows

props.conf on search head

[XmlWinEventLog:Microsoft-Windows-TerminalServices-Gateway/Operational]
rename = XmlWinEventLog

[XmlWinEventLog:Microsoft-Windows-TerminalServices-Gateway/Admin]
rename = XmlWinEventLog

[xmlwineventlog]
REPORT-0xml_kv_extract = system_props_xml_kv,system_props_xml_attributes,eventdata_xml_data,rendering_info_xml_data,userdata_xml_kv,userdata_xml_attributes

[XmlWinEventLog]
REPORT-0xml_kv_extract = system_props_xml_kv,system_props_xml_attributes,eventdata_xml_data,rendering_info_xml_data,userdata_xml_kv,userdata_xml_attributes

transforms.conf on search head

[userdata_xml_kv]
# Extracts anything in the form of <tag>value</tag> as tag::value
SOURCE_KEY = UserData_Xml
REGEX = (?ms)<(\w*)>([^<]*)<\/\1>
FORMAT = $1::$2
MV_ADD = 1

[userdata_xml_attributes]
# Extracts values from following fields:
# EventInfo: xmlns
SOURCE_KEY = UserData_Xml
REGEX = (?ms)([^\s=]+)\s*=\s*(\'[^<\']*\'|"[^<"]*")
FORMAT = $1::$2
MV_ADD = 1
Tags (1)
0 Karma

tauliang
Communicator

Assuming you have the raw XML above ingested, you should be able to parse it using SPL like below at search time:

| makeresults 
| eval _raw = "<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'>
    <System>
        <Provider Name='Microsoft-Windows-TerminalServices-Gateway' Guid='{4D5AE6A1-C7C8-4E6D-B840-4D8080B42E1B}'/>
        <EventID>303</EventID>
        <Version>0</Version>
        <Level>4</Level>
        <Task>3</Task>
        <Opcode>44</Opcode>
        <Keywords>0x4000000001000000</Keywords>
        <TimeCreated SystemTime='2020-04-16T16:16:53.376065200Z'/>
        <EventRecordID>17096</EventRecordID>
        <Correlation ActivityID='{320C9F53-4311-4E6E-8CA7-243377CB0000}'/>
        <Execution ProcessID='4676' ThreadID='13572'/>
        <Channel>Microsoft-Windows-TerminalServices-Gateway/Operational</Channel>
        <Computer>MACHINE.example.com</Computer>
        <Security UserID='S-1-5-20'/>
    </System>
    <UserData>
        <EventInfo xmlns='aag'>
            <Username>EXAMPLE\testuser</Username>
            <IpAddress>10.0.0.5</IpAddress>
            <AuthType></AuthType>
            <Resource>machine2</Resource>
            <BytesReceived>134859</BytesReceived>
            <BytesTransfered>116524</BytesTransfered>
            <SessionDuration>39</SessionDuration>
            <ConnectionProtocol>HTTP</ConnectionProtocol>
            <ErrorCode>1226</ErrorCode>
        </EventInfo>
    </UserData>
</Event>"
| spath
| rename Event.UserData.EventInfo.Username as UserName
| rename Event.UserData.EventInfo.IpAddress as IpAddress
| rename Event.UserData.EventInfo.Resource as Resource
| rename Event.UserData.EventInfo.BytesReceived as BytesReceived
| rename Event.UserData.EventInfo.BytesTransfered as BytesTransfered
| rename Event.UserData.EventInfo.SessionDuration as SessionDuration
| rename Event.UserData.EventInfo.ConnectionProtocol as ConnectionProtocol
| rename Event.UserData.EventInfo.ErrorCode as ErrorCode
| table UserName, IpAddress, Resource,BytesReceived,BytesTransfered,SessionDuration,ConnectionProtocol,ErrorCode

I don't know the details of your setup on the forwarder so couldn't comment on that.

0 Karma

ZimmermanC1
Explorer

Thanks for the response. I do not have issues parsing with SPL from search. the issue comes in when I tried to add the parsing logic to the Splunk_TA_windows add-on as a multi-value transform.

0 Karma

ZimmermanC1
Explorer

I am still stuck, any recommendations to try would be greatly appreciated.

0 Karma

hrottenberg_spl
Splunk Employee
Splunk Employee

Hi there, questions that might help frame the issue a bit better:

  1. What is the topology of your Splunk environment? Standalone, or clustered (and search vs index clusters), any intermediate tiers? If so, UF or HF? This is important so we can advise you on where these configurations need to be placed.
  2. Has the data already been successfully onboarded?
  3. Can you elaborate on what you mean by "getting it to parse with SPL"? How about a data sample, as well as the SPL?
0 Karma

ZimmermanC1
Explorer

This is a standalone splunk server with UF on each endpoint, no HF.
I have successfully collected the logs I want with a UF but cant get the fields parsed that I am trying to get out.
I am adding the configurations above to the Splunk_TA_windows/local directory

Here is an example of what one log looks like. specifically from that log I am trying to extract the fields between the UserData markup.

<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-TerminalServices-Gateway' Guid='{4D5AE6A1-C7C8-4E6D-B840-4D8080B42E1B}'/><EventID>303</EventID><Version>0</Version><Level>4</Level><Task>3</Task><Opcode>44</Opcode><Keywords>0x4000000001000000</Keywords><TimeCreated SystemTime='2020-04-16T16:16:53.376065200Z'/><EventRecordID>17096</EventRecordID><Correlation ActivityID='{320C9F53-4311-4E6E-8CA7-243377CB0000}'/><Execution ProcessID='4676' ThreadID='13572'/><Channel>Microsoft-Windows-TerminalServices-Gateway/Operational</Channel><Computer>MACHINE.example.com</Computer><Security UserID='S-1-5-20'/></System><UserData><EventInfo xmlns='aag'><Username>EXAMPLE\testuser</Username><IpAddress>10.0.0.5</IpAddress><AuthType></AuthType><Resource>machine2</Resource><BytesReceived>134859</BytesReceived><BytesTransfered>116524</BytesTransfered><SessionDuration>39</SessionDuration><ConnectionProtocol>HTTP</ConnectionProtocol><ErrorCode>1226</ErrorCode></EventInfo></UserData></Event>
0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...