How can i mask field values of AccountNum,StartAccountNum and EUDDIBAN that are extracted from xml tags . props.conf for reference

[Sujithkumarkb]

Props.conf : for field extraction
[BRM_accountnum]
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
category = Custom
pulldown_type = true
SHOULD_LINEMERGE = false
disabled = false
EXTRACT-StartAccountNum = StartAccountNum(?StartAccountNum>.+)<\/StartAccountNum>(.)
EXTRACT-EUDDIBAN = EUDDIBAN(?EUDDIBAN>.+)<\/EUDDIBAN>(.)
EXTRACT-AccountNum = AccountNum(?AccountNum>.+)<\/AccountNum>(.*)

[DavidHourani · Accepted solution]

Hi @Sujithkumarkb,

There are multiple ways to anonymize data. At index time you can do this via sedcmd :
https://docs.splunk.com/Documentation/Splunk/7.3.0/Data/Anonymizedata#Anonymize_data_with_a_sed_scri...

Or via props and transforms (it's more flexible this way):
https://docs.splunk.com/Documentation/Splunk/7.3.0/Data/Anonymizedata#Anonymize_data_with_a_regular_...

You can also anonymize at search time for hiding the results when displaying them, but that's not really anonymzation since access to the raw data would show the unmasked fields.

Cheers,
David

[Sujithkumarkb]

Thanks David for the reference .

It worked just fine for me .

cheers
Sujith

[Sujithkumarkb]

Hi @DavidHourani

I am able to extract the field and mask it , but the xml tags not showing up anymore on the events.
I am getting OrbitalConnectionUsername =###### as of now ,
expected is OrbitalConnectionUsername ######### OrbitalConnectionUsername as xml tags

Below props and transforms for reference.

inputs.conf
[monitor://path]
disabled = false
sourcetype = BRM

transforms.conf
[username-anonymizer2]
REGEX = OrbitalConnectionUsername(?OrbitalConnectionUsername.+)\/OrbitalConnectionUsername(.*)

FORMAT = $1OrbitalConnectionUsername=########$2
DEST_KEY = _raw

props.conf
[BRM]
TRANSFORMS-anonymize = username-anonymizer2,

[DavidHourani]

@Sujithkumarkb Can you please a copy of what you're getting and what your desired result is ?

[Sujithkumarkb]

raw data : tagopenOrbitalConnectionUsername >ABCD tagcloseOrbitalConnectionUsername>

Desired result is as below xml format as tags,(unable to paste >< here as tags)
expected :tag open **OrbitalConnectionUsername>########## **tag closeOrbitalConnectionUsername>

presently after the above mentioned props and transforms , i am getting data as OrbitalConnectionUsername=####### with no tags at all .

the entire raw data is an xml and to keep the format of the masked field i need OrbitalConnectionUsername also as tag as shown above.

Is there any other way i can send the raw data to you , if this is not clear ?

[Sujithkumarkb]

Hi @DavidHourani
Any suggestions on this ?

Presently i am getting data populated as below , which is not in between tags.

$AccountNum=########
OrderIDT1 18609,875D5F7D7D07939FDBBF37179FD7AC20474166545B20100B /OrderID
TxRefNum 5D5F7D7D07939FDBBF37179FD7AC20474166545B /TxRefNum

[DavidHourani]

Hi @Sujithkumarkb,

The $1 and $2 in the FORMAT line represent the capturing groups from the regex. If the capturing groups don't include the tags then they won't show in the format.

Try this as a REGEX:

REGEX = (.*\<OrbitalConnectionUsername\>).*(\<OrbitalConnectionUsername\>.*)

And this as a FORMAT:

FORMAT = $1########$2

Let me know if that helps.

[Sujithkumarkb]

@DavidHourani doesnt work ... 😞

The field extraction isnt happening nor the masking with the suggested regex

[DavidHourani]

try the regex like this :

(.*<OrbitalConnectionUsername>).*(<OrbitalConnectionUsername>.*)

Please post a sample data wrapped in "code sample" from splunk answers so I can use it to test out the regex for you.

Also apply the regex above on a sample of your data on regex101.com to make sure it's matching.

[Sujithkumarkb]

i checked both the regex and few of mine as well , extraction isnt happening
Below xml for refernce

D Tue Jul 30 11:39:01 2019 /opt/app/BRM/workspace

<?xml version="1.0" encoding="UTF-8"?>
<Response>
<NewOrderResp>
<MerchantID>111</MerchantID>
<TerminalID>001</TerminalID>
<AccountNum>XXXXXXXXXXXX7133</AccountNum>
<OrderID>T1,1822d,4</OrderID>
<OrbitalConnectionUsername>ABC123</OrbitalConnectionUsername>       
<OrbitalConnectionPassword>ZXY567</OrbitalConnectionPassword>
</NewOrderResp></Response>

[DavidHourani]

There seems to be a backslash missing in the REGEX, try it like this :

(?m)^(.*<OrbitalConnectionUsername>).*(<\/OrbitalConnectionUsername>.*)$

You can view it here :
https://regex101.com/r/SwIXOs/1

[Sujithkumarkb]

Thanks soo much David, both the solutions works perfect for me .
A Backslash played me pretty well.

Cheers 🙂

[DavidHourani]

hahah... yeah its annoying when this happens, you end up spending so much time troubleshooting something small like a backslash 🙂

Glad I could help, would really appreciate it if you can up-vote the answer and the comments, thank you !

[DavidHourani]

If the above doesn't working for you then instead of using props and transforms and use sedcmd as I mentioned initially in the answer.

It should look like this in props.conf -- already tested it and it's working :

SEDCMD-Anon= s/<OrbitalConnectionUsername>([^<])+/<OrbitalConnectionUsername>XXXXXXXXXX/g 

[gaurav_maniar]

Hi Sujithkumarkb,

Can you share some dummy data and how you want to mask the data.
It would be easy to answer with specific details.

[Sujithkumarkb]

Hi gaurav,

Below data for reference .Regex which i have used is same for all field extractions in the below xml.
Can we mask AccountNum and OrderID with "XXXs"

?xml version="1.0" encoding="UTF-8"?

Response>NewOrderResp
AccountNum 129865THGT982000/AccountNum
OrderIDT1,1828a,62/OrderID
/NewOrderResp/Response

[Sukisen1981]

hi @Sujithkumarkb - have you gone through this - https://docs.splunk.com/Documentation/Splunk/7.3.0/Data/Anonymizedata
and tried sedcmd?