Splunk Search

How to remove \ (backslash) using from URLs rex sed?

ccsfdave
Builder

I am trying to remove the escaped characters of "\" from the URLs coming in via a Twitter REST feed. Does anyone have the secret sauce for forming a rex field= mode=sed?

Sample URL: http:\/\/pbs.twimg.com\/media\/CoyGo5cUsAEmIZF.jpg

Thanks!

0 Karma
1 Solution

gabriel_vasseur
Contributor

This works for me in the search window:

| eval yourfieldname=replace(yourfieldname,"\\\\(.)","\1")

EDIT: a few words of explanation... the string "\\\\(.)" actually corresponds to the regex \\(.) which will match a single backslash followed by any character. The backslash has to be escaped once for the regex and another time to be in a double-quoted string, hence why one becomes four. If you're using the regex in a .conf file, depending how you do it, you don't need to escape it twice. Hope that helps.

NOTE: the advantage of that approach is that if your raw data has an escaped backslash (i.e. two backslashes in a row), it will do the right thing and replace it with one backslash rather than blindly removing all backslashes.

NOTE: this is probably also possible using sed.

View solution in original post

Jeremiah
Motivator

Try this:

| gentimes start=-1 | eval url="http:\/\/pbs.twimg.com\/media\/CoyGo5cUsAEmIZF.jpg"  | rex mode=sed field=url "s/\\\//g"

You may also need to use the urldecode command for some urls (|eval url=urldecode(url)).

ccsfdave
Builder

This worked as well! two ways to skin this one! thanks!

0 Karma

gabriel_vasseur
Contributor

This works for me in the search window:

| eval yourfieldname=replace(yourfieldname,"\\\\(.)","\1")

EDIT: a few words of explanation... the string "\\\\(.)" actually corresponds to the regex \\(.) which will match a single backslash followed by any character. The backslash has to be escaped once for the regex and another time to be in a double-quoted string, hence why one becomes four. If you're using the regex in a .conf file, depending how you do it, you don't need to escape it twice. Hope that helps.

NOTE: the advantage of that approach is that if your raw data has an escaped backslash (i.e. two backslashes in a row), it will do the right thing and replace it with one backslash rather than blindly removing all backslashes.

NOTE: this is probably also possible using sed.

wolanm1
Explorer

Over 4 years later and this post still helped me solve my problem! Thank you!

0 Karma

ccsfdave
Builder

this worked, thanks!

0 Karma
Get Updates on the Splunk Community!

Splunk Mobile: Your Brand-New Home Screen

Meet Your New Mobile Hub  Hello Splunk Community!  Staying connected to your data—no matter where you are—is ...

Introducing Value Insights (Beta): Understand the Business Impact your organization ...

Real progress on your strategic priorities starts with knowing the business outcomes your teams are delivering ...

Enterprise Security (ES) Essentials 8.3 is Now GA — Smarter Detections, Faster ...

As of today, Enterprise Security (ES) Essentials 8.3 is now generally available, helping SOC teams simplify ...