topic Re: Splunk ITSI: Why is transforms not masking the data correctly? in Splunk ITSI

Splunk ITSI: Why is transforms not masking the data correctly?

iamlearner123 — Mon, 22 Apr 2019 20:22:23 GMT

Hello,

I am new to splunk and learning it. However, recently i wrote transform to mask the mail ID but when i tested it is not masking the mail ID.

Transform:

[mail_id_mask]
REGEX = ([A-z0-9._%+-]+@[A-z0-9.-]+\.[A-z]{2,63})
FORMAT = ********@*********
DEST_KEY = _raw

Sample logs:

(29.2) 01-27-17 02:53:27 (9866:8500)  PRINTINGFN: $G_NOTIFY12_GRP_INTERNAL: abcdef.sdfrwe56@xyz.com
 (29.2) 01-27-17 02:53:27 (9866:8500)  PRINTINGFN: $G_NOTIF123Y_GRP_EXTERNAL: corP-apachesci.com

Any help would be appreciated.

Re: Splunk ITSI: Why is transforms not masking the data correctly?

pruthvikrishnap — Mon, 22 Apr 2019 20:56:37 GMT

try rechecking the regex and also try excluding the FORMAT and see if that works

Re: Splunk ITSI: Why is transforms not masking the data correctly?

iamlearner123 — Mon, 22 Apr 2019 22:10:16 GMT

Regex is working fine but splunk is replacing the entire event with the **********************

Thanks

Re: Splunk ITSI: Why is transforms not masking the data correctly?

ddrillic — Wed, 30 Sep 2020 00:12:56 GMT

If we look at Anonymizing Data in Splunk

You see the following -

In this approach, a TRANSFORMS statement is called in the props.conf file and is applied to the data in the queues before being indexed. In the example, the goal is to mask the “sensitive number" except for the last 4 digits.

—props.conf---
[hr_app]
TRANSFORMS-hr_app_logs_mask_data = mask_sn

—transforms.conf---
[mask_sn]
REGEX = (?m)^(.)SN=\d{3}-\d{2}-(\d{4}.)
DEST_KEY = _raw
FORMAT = $1SN=###-##-$2

This is the result of the sample event going through the transformation
“This is an event with a sensitive number in it. SN=###-##-1111. This should be masked”

The approach here is to match the first part of the event (.*), then the part to be masked (SN=…), then the last 4 digits and the rest of the event. These last two parts are to be retained when the event data is written back out to the "_raw" field specified by the "DEST_KEY." Note that the “FORMAT” setting specifies how the event will be re-written. The "$1" and "$2" refer to the two capturing groups in the "REGEX" field.

Meaning, the REGEX captures the entire event, breaks it up to multiple capturing groups and then reconstructs the event.

Re: Splunk ITSI: Why is transforms not masking the data correctly?

harsmarvania57 — Tue, 23 Apr 2019 10:02:47 GMT

Hi,

You can use SEDCMD in props.conf to achieve this easily.

Please try below config in props.conf on Heavy Forwarder/Indexer whichever comes first from Universal Forwarder.

[yoursourcetype]
SEDCMD-mailidmask = s/^(\N+[\:]\s)[^\@]+\@[^\n]+/\1XXXX/

Re: Splunk ITSI: Why is transforms not masking the data correctly?

iamlearner123 — Tue, 23 Apr 2019 17:43:09 GMT

Thank you for the reply. however, it is not masking the all the email ID's. it masked only the first email ID (abcdef.sdfrwe56@xyz.com) when i tested in regex101.

Re: Splunk ITSI: Why is transforms not masking the data correctly?

harsmarvania57 — Tue, 23 Apr 2019 17:45:11 GMT

I can't see any other email ID in raw data which you have provided.

Re: Splunk ITSI: Why is transforms not masking the data correctly?

iamlearner123 — Wed, 30 Sep 2020 00:13:15 GMT

Like, i was testing the regex for the couple of other events. Please find the below events. When i put the below events in the regex101, masking is not working. I am trying achieve a dynamic regex that will work for any email ID.

(14.2) 01-27-19 02:53:28 (8544:8500) PRINTFN: $G_NOTIFY_GRP_INTERNAL: harry.peter07@abc.com
(14.2) 01-27-18 02:53:27 (8544:8500) PRINTFN: $G_NOTIFY_GRP_EXTERNAL: SAAS-Learning_MDA@trsq.com

Thanks

Re: Splunk ITSI: Why is transforms not masking the data correctly?

harsmarvania57 — Tue, 23 Apr 2019 18:06:40 GMT

Here you go https://regex101.com/r/yHxyYg/1 , it is working fine