Splunk Search
Highlighted

How to edit my transforms.conf to drop XML event data?

Explorer

So I looked on the answer for this question and could not find it. (Look at code and sample below.) So the input is fine. It removed the top part of the XML output and the first "loggedinreps", but it never removes the last tag of "loggedinreps" . It keeps it part of the last "rep" block which is going to throw off when I try to extract the other fields.

Any help would be great!

props.conf

[source::bomgar_get_logged_in_reps]
TRANSFORMS-null = remove-xmlheader
TRANSFORMS-null = remove-logged-in-reps

[bomgar_get_logged_in_reps]
BREAK_ONLY_BEFORE = <rep[^>]+>.*?

transforms.conf

[remove-xmlheader]
REGEX = <[?]xml.*>
DEST_KEY = queue
FORMAT = nullQueue

[remove-logged-in-reps]
REGEX = <.*logged_in_reps.*>
DEST_KEY = queue
FORMAT = nullQueue

Sample Input:

<?xml version="1.0" encoding="UTF-8"?>
<logged_in_reps xmlns="http://www.bomgar.com/namespaces/API/command">
    <rep id="5">
        <display_name>User Here</display_name>
                ... Other fields removed.
        </rep>
    <rep id="6">
        <display_name>User Here</display_name>
                ... Other fields removed.
        </rep>
</logged_in_reps>
Highlighted

Re: How to edit my transforms.conf to drop XML event data?

Esteemed Legend

I suspect you are also over-consuming and deleting events, too. Try this (new REGEX):


[remove-logged-in-reps]
REGEX = </?loggedinreps>
DEST_KEY = queue
FORMAT = nullQueue

View solution in original post

Highlighted

Re: How to edit my transforms.conf to drop XML event data?

Explorer

Worked, now it's getting rid of the last "rep" pair which happens to be me in the xml output currently. arg!

0 Karma
Highlighted

Re: How to edit my transforms.conf to drop XML event data?

Esteemed Legend

It is doing this because your last event contains the end-tag (your nullqueue setting says to send any events with that string to oblivion).
If you are using LINEBRAKER, try using this instead:


LINE
BREAKER=()(\r\n)

This should prevent your "rep" events from containing the "loggedinreps" tag.

0 Karma
Highlighted

Re: How to edit my transforms.conf to drop XML event data?

Explorer

Assuming to put that in the "props.conf" under [bomgargetloggedinreps] ?

0 Karma
Highlighted

Re: How to edit my transforms.conf to drop XML event data?

Explorer

I ended up doing this:

[source::bomgar_get_logged_in_reps]
TRANSFORMS-xml = remove-xmlheader
TRANSFORMS-remove = remove-logged-in-reps

[bomgar_get_logged_in_reps]
BREAK_ONLY_BEFORE = <rep[^>]+>.*?
MUST_BREAK_AFTER = </rep>
EXTRACT-repType = (?i)<type>(?P<type>\w+)(?=<)

and that made it so the last rep block showed up. 🙂