topic Re: Anonymize only some Email Addresses in Splunk Search

Anonymize only some Email Addresses

SirHill17 — Tue, 21 Jun 2016 14:16:16 GMT

Hi,

I need help writing a regex which must anonymize email address which doesn't below to the company domain. I already did some tests but with no success. Please find below the regex I tried:

^(.*)(?:(?<!\S)(\w[\w\-\.]+@domain.com))(.*)$

So the aim of the regex is to have all the email addresses external of the company anonymized and keep in clear text the internal email addresses.

Can someone help with that?

Thanks!!!

Re: Anonymize only some Email Addresses

sundareshr — Tue, 21 Jun 2016 14:41:43 GMT

See if this works

[^@]+@(?!internaldomain)(?<e>.*)

Re: Anonymize only some Email Addresses

jkat54 — Tue, 21 Jun 2016 14:51:51 GMT

Will you do this with inline search or on ingestion?

Re: Anonymize only some Email Addresses

SirHill17 — Tue, 21 Jun 2016 14:55:55 GMT

I will setup that in transforms.conf and props.conf

Re: Anonymize only some Email Addresses

jkat54 — Tue, 21 Jun 2016 15:00:24 GMT

Since you'll use props, i recommend this multi step approach:

1st change the format of the company domain, 2nd redact every email address that matches standard email format, then change company domain back to correct format:

[sourcetype]
SEDCMD-aaa=s/@company\.org/ at company.org/g
SEDCMD-bbb=s/(\w+)@(\w+)/****@\2/g
SEDCMD-ccc=s/ at /@/g

You'll note my regex in SEDCMD-bbb is not perfect for email matching, but puts you on the right track.

Re: Anonymize only some Email Addresses

woodcock — Tue, 21 Jun 2016 15:11:42 GMT

This pages walks you through everything except building the RegEx:

http://docs.splunk.com/Documentation/Splunk/5.0/Data/Anonymizedatausingconfigurationfiles

Try this for a RegEx:

[A-z0-9._%+-]+@[A-z0-9.-]+\.[A-z]{2,63}(?<!@YourCompanyDomainHere.com)(?:[^A-z]|$)

This will match all email addresses EXCEPT for those matching anyuser@YourCompanyDomainHere.com.

Re: Anonymize only some Email Addresses

SirHill17 — Tue, 21 Jun 2016 16:25:55 GMT

Works fine!! Awesome!

Would you know how to do to use it for multiple domain names?

I tried with two SECCMD but doesn't work as the second one overwrites the first one.

Re: Anonymize only some Email Addresses

woodcock — Tue, 21 Jun 2016 16:33:55 GMT

Like this:

 [A-z0-9._%+-]+@[A-z0-9.-]+\.[A-z]{2,63}(?<!@(?:YourCompanyDomainHere.com|OtherCompany.org|AndOnAndOn.etc))(?:[^A-z]|$)

Re: Anonymize only some Email Addresses

jkat54 — Tue, 21 Jun 2016 16:35:18 GMT

The answer I gave would work fine with multiple domains but I like woodcocks regex. So try adding an or "|"

Re: Anonymize only some Email Addresses

SirHill17 — Wed, 22 Jun 2016 06:46:30 GMT

Thanks guys!

Re: Anonymize only some Email Addresses

SirHill17 — Thu, 23 Jun 2016 07:44:29 GMT

I also tested your solution, works fine as well. Thanks!

Re: Anonymize only some Email Addresses

jkat54 — Sat, 25 Jun 2016 14:06:42 GMT

I gave regex hell and couldn't make it work for nothing. Woodcocks regex works in my limited testing but I was also able to find an "all inclusive" email address detecting regex that was over 4000 characters long. So i don't believe there is a perfect one-liner regex for this problem & would recommend the multi step approach instead.

Re: Anonymize only some Email Addresses

woodcock — Sat, 25 Jun 2016 15:06:11 GMT

Yes, RegEx for email is complicated.

Re: Anonymize only some Email Addresses

jkat54 — Sat, 25 Jun 2016 15:42:41 GMT

I'm still amazed with what you crafted there!

Re: Anonymize only some Email Addresses

woodcock — Sat, 25 Jun 2016 15:45:40 GMT

It is all in the negative look-behind. RegEx is powerful, for sure.