Splunk Search
Highlighted

Masking Sensitive Data

Communicator

Ok Splunkers......

I have 1 search-head, 2 indexers, 1 Deployment server

Here is the event (sourcetype=mysourcetype)that I want to mask out the CC number:

2014-06-01 00:01:34 W3SVC1 10.0.99.120 GET /Disputes/BackToMYDomain.aspx ID=183481&ClaimNum=05/31-1370&DType=DMC&DClass=Debit%20Fraud&DeptExt=1234&**Card=1234567891234567**&SubmitDate=5/31/2014&samedayptr=N&CrdHolder=JOHN+DOE&TotClaim=150.00&Action=REVPC80MyDomain\mmouse10.XX.XX.XXMozilla/4.0+compatible;+MSIE+7.0;+Windows+NT+6.1;+WOW64;+Trident/5.0;+SLCC2;+.NET+CLR+2.0.50727;+.NET+CLR+3.5.30729;+.NET+CLR+3.0.30729;+Media+Center+PC+6.0;+.NET4.0C;+.NET4.0E;+InfoPath.3) 200 0 0

I have the following props & Transforms in my Splunk_HOME/etc/system/local/directory:

props.conf:
[sourcetype]
TRANSFORMS-1card = cc_num_anon

Transforms.conf:
[cc_num_anon]
REGEX = (.*CARD=)\d{12}(\d{4}.*)
DEST_KEY = _raw
FORMAT = $1xxxxxxxxxxxx$2

I ran the debug mode, reloaded the forwarders....but Im still seeing the card information.....Oh I forgot to mention that the variable changes from Card/CARD within the event.

0 Karma
Highlighted

Re: Masking Sensitive Data

Builder

Normally, such masking will happen at parse time on the indexer, so just to be clear, these configs are on the indexers, correct?

A few things strike me as off about your regex:

REGEX = (.CARD=)d{12}(d{4}.)

I am not sure about the dots at the beginning and end, case is not matching your data either, also the character types don't have backslashes. So, to me this looks a little better:

REGEX = (Card=)\d{12}(\d{4})

Also note that this will only apply to newly indexed data once the transform is in place.

0 Karma
Highlighted

Re: Masking Sensitive Data

Builder

See updated answer.

0 Karma
Highlighted

Re: Masking Sensitive Data

SplunkTrust
SplunkTrust

Try this props.conf-only solution that honours the variable case:

[sourcetype]
SEDCMD-cc = s/(?i)(card=)\d{12}(\d{4})/\1xxxxxxxxxxxx\2/g

This should also run considerably faster because a leading .* in a regex will eat your server's soul.

View solution in original post

Highlighted

Re: Masking Sensitive Data

Communicator

Silly question...when you say add to the props.conf...this is on the indexers, correct?

Indexer
Splunk_HOME/etc/system/local

OR

The deployment server in Deployment apps (the app) local folder.

0 Karma
Highlighted

Re: Masking Sensitive Data

Communicator

So Im having a different scenario since I added the SEDCMD to the indexer local props.conf:

Right output: (IT WORKS)

2015-01-31 03:58:10 W3SVC1 10.XX.XX.XX GET /Disputes/BackTohost.aspx ID=222888&ShID=70&
Choice=Fraud&CARD=xxxxxxxxxxxx5144&DType=DEBIT&ACode=W&SCode=G&AmtFr=&AmtTo=&DtFr=01/29/2015&DtTo=01/30/2015&CCIssued=&
Action=History 80 MyDomain\mmouse 10.XX.XX.XX Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.1;+WOW64;+Trident/5.0;+SLCC2;
+.NET+CLR+2.0.50727;+.NET+CLR+3.5.30729;+.NET+CLR+3.0.30729;+Media+Center+PC+6.0;+.NET4.0C;+.NET4.0E;+InfoPath.3) 200 0 0

Then I get a different output from the same sourcetype: (Its not changing the Card=, but adding an end output)

2015-01-30 20:33:52 W3SVC1 10.XX.XX.XX GET /Disputes/BackTohost.aspx ID=222796&
ShID=ALL%20SHARES&Choice=Fraud&CARD=1234567891234567&DType=DEBIT&ACode=W&SCode=G&AmtFr=&AmtTo=&DtFr=&DtTo=&
CCIssued=&Action=History 80 MyDoamin\mmouse 10.XX.XX.XX Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.1;+WOW64;+Trident/5.0;
+SLCC2;+.NET+CLR+2.0.50727;+.NET+CLR+3.5.30729;+.NET+CLR+3.0.30729;+Media+Center+PC+6.0;+.NET4.0C;+.NET4.0E) 200 0 0xxxxxxxxxxxx$2

The the second event it applying the numbers 0xxxxxxxxxxxx$2

Is this because the event is different in word count?

0 Karma
Highlighted

Re: Masking Sensitive Data

SplunkTrust
SplunkTrust

This config belongs on the indexers. If you have heavy forwarders that perform parsing then you will need it there as well.

The second event is several hours older, it was indexed before the SEDCMD was applied. I'm sure of this because there is $2 at the end, and there is no dollar sign used in the SEDCMD.

0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.