topic Re: Masking SSN info into splunk at index time in Getting Data In

Masking SSN info into splunk at index time

mike7860 — Fri, 25 Jan 2013 14:24:11 GMT

I would like to know how to mask SSN information in splunk at index time. There is a documentation available on the splunk homepage. But the question arises that whether configurations of props.conf and transforms.conf be done at the indexer side as we have a multi-tiered splunk network? Then, the other question is how do I compile a rex expression for masking SSN data from the followin log source:

index=web sourcetype=weblogic_app ssn

/env:Header
env:Body

ns2:ContractsWithProductInfoResponse
ns2:ParticipantDetails
*ns2:Pin2895237/ns2:Pin
ns2:SSN/ns2:SSN*

Re: Masking SSN info into splunk at index time

rgcurry — Fri, 25 Jan 2013 15:44:25 GMT

Check out the Splunk Doc page at

http://docs.splunk.com/Documentation/Splunk/4.2.1/Data/Anonymizedatausingconfigurationfiles

It includes a SED example that fits closely to what you are wanting to do. The only difference is your data is XML formatted versus the 'straight' text of the doc example but that is not big deal to deal with in the RegEx you'd use to pinpoint your data to mask.

Re: Masking SSN info into splunk at index time

johnhsu — Fri, 07 Feb 2014 17:41:10 GMT

Regex for masking Social Security Number in XML TAG

Result:

<relatedPolicies />
<socialSecurityNumber>###-##-1234</socialSecurityNumber>
<ssnRefusedIndicator>false</ssnRefusedIndicator>

transforms.conf:

[SSN-anonymizer]
REGEX = (?ms)^(.*)\<[sS]ocialSecurityNumber>\d{3}-?\d{2}-?(\d{4}.*)$
FORMAT = $1<socialSecurityNumber>###-##-$2
DEST_KEY = _raw

Note:
In Splunk V6.0.1
In my testing, masking function failed if the record is very large, for example 15KB, and SSB is out of 100 lines (or some position) .

Thanks
Sincerely
John Hsu

Re: Masking SSN info into splunk at index time

johnhsu — Fri, 07 Feb 2014 18:06:57 GMT

Corrected: Don't know why missed the star "" after those two dot in post
Try post again:
"REGEX = (?ms)^(.)<[sS]ocialSecurityNumber>\d{3}-?\d{2}-?(\d{4}.*)$"

Thanks
Sincerely
John Hsu

Re: Masking SSN info into splunk at index time

johnhsu — Fri, 07 Feb 2014 18:23:14 GMT

Sorry! Try again

should be

REGEX = (?ms)^(.\*)\<[sS]ocialSecurityNumber>\d{3}-?\d{2}-?(\d{4}.\*)$

Thanks
Sincerely
John Hsu

Re: Masking SSN info into splunk at index time

Lowell — Thu, 03 Jul 2014 15:06:26 GMT

Johnsu, just so you know. You can edit your own answers. That's preferred over posting new ones. Thanks for taking the time to post your corrections!

Re: Masking SSN info into splunk at index time

yannK — Wed, 15 Oct 2014 16:42:35 GMT

try the sedcmd command in the props.conf, and deploy on your parsing instances :
see http://docs.splunk.com/Documentation/Splunk/6.1.4/Data/Anonymizedatausingconfigurationfiles

on all indexers
on all heavy forwarders
in case of csv/iis or INDEXED_EXTRACTION sourcetypes (since splunk 6.1), on the forwarders (even the Universal and Lightweight)

[mysourcetype]
SEDCMD-hidessn=s/(GovernmentIDNumber: SSN: \d{3})(\d{6})/\1xxxxxxx/g

GovernmentIDNumber: SSN: 555666666
-> GovernmentIDNumber: SSN: 555xxxxxxx