I have to ingest some data so i've created a field called customer data and the regex works fine - ^[0-9]{16}.{249}(?<customer_information>.{174}).
As it contains PII data i need to mask it but keep the format of that event so the 174 characters within the customer_information field news to show as ####
Ive created this within the props.conf file but I can't get the data to be shown as ###. can you help?
[mask_customer_data]
DEST_KEY = _raw
REGEX = ^[0-9]{16}.{249}(?<customer_information>.{174})
FORMAT = $1CI##############################################################################################################################################################################
Hi @gcusello,
I found a solution which is the regex below. Thanks for your help though ! 🙂
REGEX = (^[0-9]{16}.{249}).{174}(.+)
FORMAT = $1##############################################################################################################################################################################$2
DEST_KEY = _raw
Hi @vishalduttauk,
your regex is to extract a field not to replace a part of an event with a masquerading string, you can find more information about data masking at https://docs.splunk.com/Documentation/Splunk/latest/Data/Anonymizedata
so please trysomething like this
props.conf:
[your_sourcetype]
TRANSFORMS-anonymizer = session-anonymizer
transforms.conf:
[session-anonymizer]
REGEX = ^[0-9]{16}.{249}(.{174})
FORMAT = ^[0-9]{16}.{249}(\*{174})
DEST_KEY = _raw
Ciao.
Giuseppe
Hi Giuseppe,
Thanks for your reply. I updated the props.conf file and created the transforms.conf file as per your reply. I have tried uploading some sample data (csv) and this is what happens:
Hi @gcusello ,
Thanks for your help so far but its not working yet. I have attached a sample record if that helps?
Hi @gcusello,
I used the html editor to paste in an example and keep the format of the text:
0000440987654321 2901012001000000000002650380000767 2 000000 000000 00000000 VMr Vishal S Dutta 001 Street Road Town AA11AA 000000001011999 000000000 00000000 0000000M
Hi @vishalduttauk,
let me understand, you want as output:
0000440987654321 2901012001000000000002650380000767 2 000000 000000 00000000 ***********************************************************************************************************************************************************************
is it correct?
In this case, please try this:
REGEX = ^((\d+\s+){6})(.*)
FORMAT = ^($1)(\*{174})
Ciao.
Giuseppe
Ciao.
Giuseppe
Hi @gcusello
This is the format:
0000440987654321 2901012001000000000002650380000767 2 000000 000000 00000000 M############################################################################################################################################################################# 000000001011999 000000000 00000000 0000000M
Hi @vishalduttauk,
please try this:
REGEX = ^((\d+\s+){6})(.*)(\s+\d+\s+\d+\s+\d+\s+\w+)
FORMAT = ^($1)(\#{174})($3)
Ciao.
Giuseppe
Hi @gcusello
This is what I see now:
Could something within the the props file that is causing issues?
This is what i have set up
[CRA_Consumer_Txt_data]
CHARSET = UTF-8
DATETIME_CONFIG = CURRENT
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
TRANSFORMS-anonymizer = session-anonymizer
SHOULD_LINEMERGE = false
category = Custom
disabled = false
pulldown_type = true
EXTRACT-close_date = ^[0-9]{16}.{4}[0-9]{2}[0-9]{8}(?<close_date>[0-9]{8})[0-9]{9}[0-9]{7}.{1}[0-9].{208}[ A-Z].{182}[0-9]{8}.{13}[0-9]{8}.{26}[0-9]{7}[A-Z]
EXTRACT-current_balance = ^[0-9]{16}.{4}[0-9]{2}[0-9]{8}[0-9]{8}[0-9]{9}(?<current_balance>[0-9]{7}).{1}[0-9].{208}[ A-Z].{182}[0-9]{8}.{13}[0-9]{8}.{26}[0-9]{7}[A-Z]
EXTRACT-status = ^[0-9]{16}.{4}[0-9]{2}[0-9]{8}[0-9]{8}[0-9]{9}[0-9]{7}.{1}(?<status>[0-9]).{208}[ A-Z].{182}[0-9]{8}.{13}[0-9]{8}.{26}[0-9]{7}[A-Z]
EXTRACT-flag = ^[0-9]{16}.{4}[0-9]{2}[0-9]{8}[0-9]{8}[0-9]{9}[0-9]{7}.{1}[0-9].{208}(?<flag>[ A-Z]).{182}[0-9]{8}.{13}[0-9]{8}.{26}[0-9]{7}[A-Z]
EXTRACT-dob = ^[0-9]{16}.{4}[0-9]{2}[0-9]{8}[0-9]{8}[0-9]{9}[0-9]{7}.{1}[0-9].{208}[ A-Z].{182}(?<dob>[0-9]{8}).{13}[0-9]{8}.{26}[0-9]{7}[A-Z]
EXTRACT-start_date = ^[0-9]{16}.{4}[0-9]{2}(?<start_date>[0-9]{8})[0-9]{8}[0-9]{9}[0-9]{7}.{1}[0-9].{208}[ A-Z].{182}[0-9]{8}.{13}[0-9]{8}.{26}[0-9]{7}[A-Z]
EXTRACT-payment_type = ^[0-9]{16}.{4}[0-9]{2}[0-9]{8}[0-9]{8}[0-9]{9}[0-9]{7}.{1}[0-9].{208}[ A-Z].{182}[0-9]{8}.{13}[0-9]{8}.{26}(?<payment_type>[0-9]{7}[A-Z])
EXTRACT-customer_information = ^[0-9]{16}.{249}(?<customer_information>.{174})
Hi @gcusello,
I found a solution which is the regex below. Thanks for your help though ! 🙂
REGEX = (^[0-9]{16}.{249}).{174}(.+)
FORMAT = $1##############################################################################################################################################################################$2
DEST_KEY = _raw