All Apps and Add-ons

Website input: How to remove backslashes from raw data?

danesen
Explorer

Hi Splunkers,

I'm using the Website input app to scrape a page that's presented in XML. The issue is that the output contains backslashes which is not there when you load the web page in a browser. It looks like it's being put there to escape the quotes
\"info\"

Is there a way to remove the backslash?

Thanks,
dan

0 Karma
1 Solution

danesen
Explorer

Resolved it. Thanks jkat54 and others for pointing me in the right direction.

Updated the props.conf file with:

[source::web_input://TestNode1]
SEDCMD-RemovingBackSlash = s/\\//g

NOTE: if you try to do this in SPL. You need 3 backslashes. It's a known thing apparently e.g.: https://answers.splunk.com/answers/24026/sedcmd-special-requirement-for-backslash.html

View solution in original post

0 Karma

wrangler2x
Motivator

I've tried a bunch of different things, including all the ones suggested here, and none of them work (I'm running on 6.1.5, by the way). I can get them working in mode=sed in a splunk search rex, but not in SEDCMD. This, however totally works in SEDCMD:

SEDCMD-RemoveBackslashes = s/\x5C{1}//g

The \x5C by itself works in ... | rex mode=sed "s/\x5C//g" but not in SEDCMD for whatever reason. Adding the {1} was just a shot in the dark, but it works. It also works in search with mode=sed and rex.

0 Karma

danesen
Explorer

Resolved it. Thanks jkat54 and others for pointing me in the right direction.

Updated the props.conf file with:

[source::web_input://TestNode1]
SEDCMD-RemovingBackSlash = s/\\//g

NOTE: if you try to do this in SPL. You need 3 backslashes. It's a known thing apparently e.g.: https://answers.splunk.com/answers/24026/sedcmd-special-requirement-for-backslash.html

View solution in original post

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

If you KNOW that ALL backslashes need to be removed, then you can easily remove them all. But that assumes that you want ALL the backslashes removed. If there are instances of backslashes which you want to keep, then you have to plan for that.

Removing backslashes only from before a quote might be what you want, but what if you have \"? Please provide some additional information about exactly what you need removed (and possibly why). That way he answer you get can be correct, and not "correct if and only if."

0 Karma

danesen
Explorer

I just want to remove ALL backslashes.

0 Karma

LukeMurphey
Champion

Can you provide an example of the entire event (the _raw field)?

0 Karma

danesen
Explorer

Because it's in XML, I'm having a hard time posting it here without it being interpreted. Once I click submit it removes all the tags...etc.

0 Karma

danesen
Explorer

Here's a sample example
semaphores available=\"4\"
Returns field semaphores available=\

0 Karma

jkat54
SplunkTrust
SplunkTrust

Have you tried SEDCMD-anyNameYouLike = s/\\"/"/g in your props.conf?

Have you tried ... | rex "s/\\"/"/g" in your search?

danesen
Explorer

Unfortunately that regex doesn't work. Nor a modified one where I try to capture the backslash:

index=* |rex mode=sed "s/\\"/\"/g"

Doesn't remove the backslash.

0 Karma

danesen
Explorer

OK. This works.
index=* | rex mode=sed "s/\\"/"/g"
Still giving backslashes in my 'Interesting Fields', but I'm closer to what I need now. Will work on the props.conf.

Thanks!

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.