Splunk Search

How to remove \ (backslash) using from URLs rex sed?

ccsfdave
Builder

I am trying to remove the escaped characters of "\" from the URLs coming in via a Twitter REST feed. Does anyone have the secret sauce for forming a rex field= mode=sed?

Sample URL: http:\/\/pbs.twimg.com\/media\/CoyGo5cUsAEmIZF.jpg

Thanks!

0 Karma
1 Solution

gabriel_vasseur
Contributor

This works for me in the search window:

| eval yourfieldname=replace(yourfieldname,"\\\\(.)","\1")

EDIT: a few words of explanation... the string "\\\\(.)" actually corresponds to the regex \\(.) which will match a single backslash followed by any character. The backslash has to be escaped once for the regex and another time to be in a double-quoted string, hence why one becomes four. If you're using the regex in a .conf file, depending how you do it, you don't need to escape it twice. Hope that helps.

NOTE: the advantage of that approach is that if your raw data has an escaped backslash (i.e. two backslashes in a row), it will do the right thing and replace it with one backslash rather than blindly removing all backslashes.

NOTE: this is probably also possible using sed.

View solution in original post

Jeremiah
Motivator

Try this:

| gentimes start=-1 | eval url="http:\/\/pbs.twimg.com\/media\/CoyGo5cUsAEmIZF.jpg"  | rex mode=sed field=url "s/\\\//g"

You may also need to use the urldecode command for some urls (|eval url=urldecode(url)).

ccsfdave
Builder

This worked as well! two ways to skin this one! thanks!

0 Karma

gabriel_vasseur
Contributor

This works for me in the search window:

| eval yourfieldname=replace(yourfieldname,"\\\\(.)","\1")

EDIT: a few words of explanation... the string "\\\\(.)" actually corresponds to the regex \\(.) which will match a single backslash followed by any character. The backslash has to be escaped once for the regex and another time to be in a double-quoted string, hence why one becomes four. If you're using the regex in a .conf file, depending how you do it, you don't need to escape it twice. Hope that helps.

NOTE: the advantage of that approach is that if your raw data has an escaped backslash (i.e. two backslashes in a row), it will do the right thing and replace it with one backslash rather than blindly removing all backslashes.

NOTE: this is probably also possible using sed.

wolanm1
Explorer

Over 4 years later and this post still helped me solve my problem! Thank you!

0 Karma

ccsfdave
Builder

this worked, thanks!

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Think Like an Architect: Introducing the Splunk Certified Cybersecurity Defense ...

In cybersecurity, defenders respond to threats. Architects design the systems that stop them.    As ...

Best Practices: Splunk auto adjust pipeline queue

When you enable autoAdjustQueue in Splunk, maxSize should be understood as the queue size Splunk starts with ...

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...