Splunk Search

Masking Sensitive Data

dperry
Communicator

Ok Splunkers......

I have 1 search-head, 2 indexers, 1 Deployment server

Here is the event (sourcetype=mysourcetype)that I want to mask out the CC number:

2014-06-01 00:01:34 W3SVC1 10.0.99.120 GET /Disputes/BackToMYDomain.aspx ID=183481&ClaimNum=05/31-1370&DType=DMC&DClass=Debit%20Fraud&DeptExt=1234&**Card=1234567891234567**&SubmitDate=5/31/2014&samedayptr=N&CrdHolder=JOHN+DOE&TotClaim=150.00&Action=REVPC80MyDomain\mmouse10.XX.XX.XXMozilla/4.0+compatible;+MSIE+7.0;+Windows+NT+6.1;+WOW64;+Trident/5.0;+SLCC2;+.NET+CLR+2.0.50727;+.NET+CLR+3.5.30729;+.NET+CLR+3.0.30729;+Media+Center+PC+6.0;+.NET4.0C;+.NET4.0E;+InfoPath.3) 200 0 0

I have the following props & Transforms in my Splunk_HOME/etc/system/local/directory:

props.conf:
[sourcetype]
TRANSFORMS-1card = cc_num_anon

Transforms.conf:
[cc_num_anon]
REGEX = (.*CARD=)\d{12}(\d{4}.*)
DEST_KEY = _raw
FORMAT = $1xxxxxxxxxxxx$2

I ran the debug mode, reloaded the forwarders....but Im still seeing the card information.....Oh I forgot to mention that the variable changes from Card/CARD within the event.

0 Karma
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

Try this props.conf-only solution that honours the variable case:

[sourcetype]
SEDCMD-cc = s/(?i)(card=)\d{12}(\d{4})/\1xxxxxxxxxxxx\2/g

This should also run considerably faster because a leading .* in a regex will eat your server's soul.

View solution in original post

martin_mueller
SplunkTrust
SplunkTrust

Try this props.conf-only solution that honours the variable case:

[sourcetype]
SEDCMD-cc = s/(?i)(card=)\d{12}(\d{4})/\1xxxxxxxxxxxx\2/g

This should also run considerably faster because a leading .* in a regex will eat your server's soul.

martin_mueller
SplunkTrust
SplunkTrust

This config belongs on the indexers. If you have heavy forwarders that perform parsing then you will need it there as well.

The second event is several hours older, it was indexed before the SEDCMD was applied. I'm sure of this because there is $2 at the end, and there is no dollar sign used in the SEDCMD.

0 Karma

dperry
Communicator

So Im having a different scenario since I added the SEDCMD to the indexer local props.conf:

Right output: (IT WORKS)

2015-01-31 03:58:10 W3SVC1 10.XX.XX.XX GET /Disputes/BackTohost.aspx ID=222888&ShID=70&
Choice=Fraud&CARD=xxxxxxxxxxxx5144&DType=DEBIT&ACode=W&SCode=G&AmtFr=&AmtTo=&DtFr=01/29/2015&DtTo=01/30/2015&CCIssued=&
Action=History 80 MyDomain\mmouse 10.XX.XX.XX Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.1;+WOW64;+Trident/5.0;+SLCC2;
+.NET+CLR+2.0.50727;+.NET+CLR+3.5.30729;+.NET+CLR+3.0.30729;+Media+Center+PC+6.0;+.NET4.0C;+.NET4.0E;+InfoPath.3) 200 0 0

Then I get a different output from the same sourcetype: (Its not changing the Card=, but adding an end output)

2015-01-30 20:33:52 W3SVC1 10.XX.XX.XX GET /Disputes/BackTohost.aspx ID=222796&
ShID=ALL%20SHARES&Choice=Fraud&CARD=1234567891234567&DType=DEBIT&ACode=W&SCode=G&AmtFr=&AmtTo=&DtFr=&DtTo=&
CCIssued=&Action=History 80 MyDoamin\mmouse 10.XX.XX.XX Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.1;+WOW64;+Trident/5.0;
+SLCC2;+.NET+CLR+2.0.50727;+.NET+CLR+3.5.30729;+.NET+CLR+3.0.30729;+Media+Center+PC+6.0;+.NET4.0C;+.NET4.0E) 200 0 0xxxxxxxxxxxx$2

The the second event it applying the numbers 0xxxxxxxxxxxx$2

Is this because the event is different in word count?

0 Karma

dperry
Communicator

Silly question...when you say add to the props.conf...this is on the indexers, correct?

Indexer
Splunk_HOME/etc/system/local

OR

The deployment server in Deployment apps (the app) local folder.

0 Karma

chanfoli
Builder

Normally, such masking will happen at parse time on the indexer, so just to be clear, these configs are on the indexers, correct?

A few things strike me as off about your regex:

REGEX = (.CARD=)d{12}(d{4}.)

I am not sure about the dots at the beginning and end, case is not matching your data either, also the character types don't have backslashes. So, to me this looks a little better:

REGEX = (Card=)\d{12}(\d{4})

Also note that this will only apply to newly indexed data once the transform is in place.

0 Karma

chanfoli
Builder

See updated answer.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...