topic Re: How to edit my transforms.conf to drop XML event data? in Splunk Search

How to edit my transforms.conf to drop XML event data?

ShaneF — Mon, 28 Sep 2020 19:47:29 GMT

So I looked on the answer for this question and could not find it. (Look at code and sample below.) So the input is fine. It removed the top part of the XML output and the first "logged_in_reps", but it never removes the last tag of "logged_in_reps" . It keeps it part of the last "rep" block which is going to throw off when I try to extract the other fields.

Any help would be great!

props.conf

[source::bomgar_get_logged_in_reps]
TRANSFORMS-null = remove-xmlheader
TRANSFORMS-null = remove-logged-in-reps

[bomgar_get_logged_in_reps]
BREAK_ONLY_BEFORE = <rep[^>]+>.*?

transforms.conf

[remove-xmlheader]
REGEX = <[?]xml.*>
DEST_KEY = queue
FORMAT = nullQueue

[remove-logged-in-reps]
REGEX = <.*logged_in_reps.*>
DEST_KEY = queue
FORMAT = nullQueue

Sample Input:

<?xml version="1.0" encoding="UTF-8"?>
<logged_in_reps xmlns="http://www.bomgar.com/namespaces/API/command">
    <rep id="5">
        <display_name>User Here</display_name>
                ... Other fields removed.
        </rep>
    <rep id="6">
        <display_name>User Here</display_name>
                ... Other fields removed.
        </rep>
</logged_in_reps>

Re: How to edit my transforms.conf to drop XML event data?

woodcock — Mon, 28 Sep 2020 19:47:31 GMT

I suspect you are also over-consuming and deleting events, too. Try this (new REGEX):



[remove-logged-in-reps]

REGEX = </?logged_in_reps>

DEST_KEY = queue

FORMAT = nullQueue

Re: How to edit my transforms.conf to drop XML event data?

ShaneF — Thu, 30 Apr 2015 13:15:47 GMT

Worked, now it's getting rid of the last "rep" pair which happens to be me in the xml output currently. arg!

Re: How to edit my transforms.conf to drop XML event data?

woodcock — Mon, 28 Sep 2020 19:47:56 GMT

It is doing this because your last event contains the end-tag (your nullqueue setting says to send any events with that string to oblivion).
If you are using LINE_BRAKER, try using this instead:



LINE_BREAKER=()(\r\n)

This should prevent your "rep" events from containing the "logged_in_reps" tag.

Re: How to edit my transforms.conf to drop XML event data?

ShaneF — Mon, 28 Sep 2020 19:47:58 GMT

Assuming to put that in the "props.conf" under [bomgar_get_logged_in_reps] ?

Re: How to edit my transforms.conf to drop XML event data?

ShaneF — Thu, 30 Apr 2015 14:13:05 GMT

I ended up doing this:

[source::bomgar_get_logged_in_reps]
TRANSFORMS-xml = remove-xmlheader
TRANSFORMS-remove = remove-logged-in-reps

[bomgar_get_logged_in_reps]
BREAK_ONLY_BEFORE = <rep[^>]+>.*?
MUST_BREAK_AFTER = </rep>
EXTRACT-repType = (?i)<type>(?P<type>\w+)(?=<)

and that made it so the last rep block showed up. 🙂