Splunk Search

Masking data using regex during Indexing

nirmalya2006
Path Finder

Hi All

I am trying to mask account numbers at indexing.
So I have the respective entries in props.conf and transforms.conf

The transforms.conf entry looks like this

REGEX = ^(.*)([+,\s,=,A-Z])37\d{9}(.*)$
FORMAT = $1$237#########$3
DEST_KEY = _raw

This is working fine for masking except that it is removing a part of the matched pattern.
So my log entry is as below

sysISN=0104B382&TRN=0010FDF1&pf=SYSTEM&gxn=ACCOUNT&gxf=37123456789 HTTP/1.1" 200 31513 112258   

After masking the entry is something below

sysISN=0104B382&TRN=0010FDF1&pf=SYSTEM&gxn=ACCOUNT&gxf7######### HTTP/1.1" 200 31513 112258

So though ideally the value should have been =37########, the masked value leaves out the =3
And this happens for all the combinations. The value of $2 and 3 from teh acoount number gets removed at masking.

Can any one help me identify the fault and resolve it.

Thanks

0 Karma
1 Solution

DMohn
Motivator

If you want to mask everything that starts with =37 (no matter what the field is named) you can use:

REGEX = ^(.*)(=37)\d{9}(.*)$
FORMAT = $1$2#########$3
DEST_KEY=_raw

If the key for the field you want to mask is always named gxfyou should include that in the regex like this:

REGEX = ^(.*)(gxf=37)\d{9}(.*)$

View solution in original post

0 Karma

DMohn
Motivator

If you want to mask everything that starts with =37 (no matter what the field is named) you can use:

REGEX = ^(.*)(=37)\d{9}(.*)$
FORMAT = $1$2#########$3
DEST_KEY=_raw

If the key for the field you want to mask is always named gxfyou should include that in the regex like this:

REGEX = ^(.*)(gxf=37)\d{9}(.*)$
0 Karma

nirmalya2006
Path Finder

That is not always. Coz I have other data like below :
TRI+37123456789
acct 37123456789
FLG37123456789

This is why I cannot stick to =37 always.
I wanted one regex that will support all of the above data.

0 Karma

DMohn
Motivator

Okay, in that case you can (almost) use your original RegEx, just put the 37 in the second capturing group:

REGEX = ^(.*)([+,\s,=,A-Z]37)\d{9}(.*)$
FORMAT = $1$2#########$3

nirmalya2006
Path Finder

Yeah... That worked.. can you please put this as answer
Thank you so much

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...