topic Re: Masking data using regex during Indexing in Splunk Search

Masking data using regex during Indexing

nirmalya2006 — Wed, 12 Apr 2017 12:03:16 GMT

Hi All

I am trying to mask account numbers at indexing.
So I have the respective entries in props.conf and transforms.conf

The transforms.conf entry looks like this

REGEX = ^(.*)([+,\s,=,A-Z])37\d{9}(.*)$
FORMAT = $1$237#########$3
DEST_KEY = _raw

This is working fine for masking except that it is removing a part of the matched pattern.
So my log entry is as below

sysISN=0104B382&TRN=0010FDF1&pf=SYSTEM&gxn=ACCOUNT&gxf=37123456789 HTTP/1.1" 200 31513 112258

After masking the entry is something below

sysISN=0104B382&TRN=0010FDF1&pf=SYSTEM&gxn=ACCOUNT&gxf7######### HTTP/1.1" 200 31513 112258

So though ideally the value should have been =37########, the masked value leaves out the =3
And this happens for all the combinations. The value of $2 and 3 from teh acoount number gets removed at masking.

Can any one help me identify the fault and resolve it.

Thanks

Re: Masking data using regex during Indexing

DMohn — Wed, 12 Apr 2017 12:18:11 GMT

If you want to mask everything that starts with =37 (no matter what the field is named) you can use:

REGEX = ^(.*)(=37)\d{9}(.*)$
FORMAT = $1$2#########$3
DEST_KEY=_raw

If the key for the field you want to mask is always named gxfyou should include that in the regex like this:

REGEX = ^(.*)(gxf=37)\d{9}(.*)$

Re: Masking data using regex during Indexing

nirmalya2006 — Wed, 12 Apr 2017 12:37:08 GMT

That is not always. Coz I have other data like below :
TRI+37123456789
acct 37123456789
FLG37123456789

This is why I cannot stick to =37 always.
I wanted one regex that will support all of the above data.

Re: Masking data using regex during Indexing

DMohn — Wed, 12 Apr 2017 13:28:45 GMT

Okay, in that case you can (almost) use your original RegEx, just put the 37 in the second capturing group:

REGEX = ^(.*)([+,\s,=,A-Z]37)\d{9}(.*)$
FORMAT = $1$2#########$3

Re: Masking data using regex during Indexing

nirmalya2006 — Wed, 12 Apr 2017 13:48:50 GMT

Yeah... That worked.. can you please put this as answer
Thank you so much