How to edit my transforms.conf to drop XML event data?

[ShaneF]

So I looked on the answer for this question and could not find it. (Look at code and sample below.) So the input is fine. It removed the top part of the XML output and the first "logged_in_reps", but it never removes the last tag of "logged_in_reps" . It keeps it part of the last "rep" block which is going to throw off when I try to extract the other fields.

Any help would be great!

props.conf

[source::bomgar_get_logged_in_reps]
TRANSFORMS-null = remove-xmlheader
TRANSFORMS-null = remove-logged-in-reps

[bomgar_get_logged_in_reps]
BREAK_ONLY_BEFORE = <rep[^>]+>.*?

transforms.conf

[remove-xmlheader]
REGEX = <[?]xml.*>
DEST_KEY = queue
FORMAT = nullQueue

[remove-logged-in-reps]
REGEX = <.*logged_in_reps.*>
DEST_KEY = queue
FORMAT = nullQueue

Sample Input:

<?xml version="1.0" encoding="UTF-8"?>
<logged_in_reps xmlns="http://www.bomgar.com/namespaces/API/command">
    <rep id="5">
        <display_name>User Here</display_name>
                ... Other fields removed.
        </rep>
    <rep id="6">
        <display_name>User Here</display_name>
                ... Other fields removed.
        </rep>
</logged_in_reps>

[woodcock · Accepted solution]

I suspect you are also over-consuming and deleting events, too. Try this (new REGEX):

[remove-logged-in-reps]

REGEX = </?logged_in_reps>

DEST_KEY = queue

FORMAT = nullQueue

[ShaneF]

Worked, now it's getting rid of the last "rep" pair which happens to be me in the xml output currently. arg!

[woodcock]

It is doing this because your last event contains the end-tag (your nullqueue setting says to send any events with that string to oblivion).
If you are using LINE_BRAKER, try using this instead:

LINE_BREAKER=()(\r\n)

This should prevent your "rep" events from containing the "logged_in_reps" tag.

[ShaneF]

Assuming to put that in the "props.conf" under [bomgar_get_logged_in_reps] ?

[ShaneF]

I ended up doing this:

[source::bomgar_get_logged_in_reps]
TRANSFORMS-xml = remove-xmlheader
TRANSFORMS-remove = remove-logged-in-reps

[bomgar_get_logged_in_reps]
BREAK_ONLY_BEFORE = <rep[^>]+>.*?
MUST_BREAK_AFTER = </rep>
EXTRACT-repType = (?i)<type>(?P<type>\w+)(?=<)

and that made it so the last rep block showed up. 🙂