topic Re: How can i mask field values of AccountNum,StartAccountNum and EUDDIBAN that are extracted from xml tags . props.conf for reference in Security

How can i mask field values of AccountNum,StartAccountNum and EUDDIBAN that are extracted from xml tags . props.conf for reference

Sujithkumarkb — Wed, 30 Sep 2020 01:35:02 GMT

Props.conf : for field extraction
[BRM_accountnum]
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
category = Custom
pulldown_type = true
SHOULD_LINEMERGE = false
disabled = false
EXTRACT-StartAccountNum = StartAccountNum(?StartAccountNum>.+)<\/StartAccountNum>(.)
EXTRACT-EUDDIBAN = EUDDIBAN(?EUDDIBAN>.+)<\/EUDDIBAN>(.)
EXTRACT-AccountNum = AccountNum(?AccountNum>.+)<\/AccountNum>(.*)

Re: How can i mask field values of AccountNum,StartAccountNum and EUDDIBAN that are extracted from xml tags . props.conf for reference

gaurav_maniar — Fri, 02 Aug 2019 10:28:49 GMT

Hi Sujithkumarkb,

Can you share some dummy data and how you want to mask the data.
It would be easy to answer with specific details.

Re: How can i mask field values of AccountNum,StartAccountNum and EUDDIBAN that are extracted from xml tags . props.conf for reference

Sujithkumarkb — Fri, 02 Aug 2019 10:43:12 GMT

Hi gaurav,

Below data for reference .Regex which i have used is same for all field extractions in the below xml.
Can we mask AccountNum and OrderID with "XXXs"

?xml version="1.0" encoding="UTF-8"?

Response>NewOrderResp
AccountNum 129865THGT982000/AccountNum
OrderIDT1,1828a,62/OrderID
/NewOrderResp/Response

Re: How can i mask field values of AccountNum,StartAccountNum and EUDDIBAN that are extracted from xml tags . props.conf for reference

Sukisen1981 — Sat, 03 Aug 2019 07:30:46 GMT

hi @Sujithkumarkb - have you gone through this - https://docs.splunk.com/Documentation/Splunk/7.3.0/Data/Anonymizedata
and tried sedcmd?

Re: How can i mask field values of AccountNum,StartAccountNum and EUDDIBAN that are extracted from xml tags . props.conf for reference

DavidHourani — Sun, 04 Aug 2019 08:15:43 GMT

Hi @Sujithkumarkb,

There are multiple ways to anonymize data. At index time you can do this via sedcmd :
https://docs.splunk.com/Documentation/Splunk/7.3.0/Data/Anonymizedata#Anonymize_data_with_a_sed_script

Or via props and transforms (it's more flexible this way):
https://docs.splunk.com/Documentation/Splunk/7.3.0/Data/Anonymizedata#Anonymize_data_with_a_regular_expression_transform

You can also anonymize at search time for hiding the results when displaying them, but that's not really anonymzation since access to the raw data would show the unmasked fields.

Cheers,
David

Re: How can i mask field values of AccountNum,StartAccountNum and EUDDIBAN that are extracted from xml tags . props.conf for reference

Sujithkumarkb — Mon, 05 Aug 2019 06:59:02 GMT

Thanks David for the reference .

It worked just fine for me .

cheers
Sujith

Re: How can i mask field values of AccountNum,StartAccountNum and EUDDIBAN that are extracted from xml tags . props.conf for reference

Sujithkumarkb — Thu, 22 Aug 2019 11:58:48 GMT

Hi @DavidHourani

I am able to extract the field and mask it , but the xml tags not showing up anymore on the events.
I am getting OrbitalConnectionUsername =###### as of now ,
expected is OrbitalConnectionUsername ######### OrbitalConnectionUsername as xml tags

Below props and transforms for reference.

inputs.conf
[monitor://path]
disabled = false
sourcetype = BRM

transforms.conf
[username-anonymizer2]
REGEX = OrbitalConnectionUsername(?OrbitalConnectionUsername.+)\/OrbitalConnectionUsername(.*)

FORMAT = $1OrbitalConnectionUsername=########$2
DEST_KEY = _raw

props.conf
[BRM]
TRANSFORMS-anonymize = username-anonymizer2,

Re: How can i mask field values of AccountNum,StartAccountNum and EUDDIBAN that are extracted from xml tags . props.conf for reference

DavidHourani — Thu, 22 Aug 2019 12:18:11 GMT

@Sujithkumarkb Can you please a copy of what you're getting and what your desired result is ?

Re: How can i mask field values of AccountNum,StartAccountNum and EUDDIBAN that are extracted from xml tags . props.conf for reference

Sujithkumarkb — Wed, 30 Sep 2020 01:51:46 GMT

raw data : tagopenOrbitalConnectionUsername >ABCD tagcloseOrbitalConnectionUsername>

Desired result is as below xml format as tags,(unable to paste >< here as tags)
expected :tag open **OrbitalConnectionUsername>########## **tag closeOrbitalConnectionUsername>

presently after the above mentioned props and transforms , i am getting data as OrbitalConnectionUsername=####### with no tags at all .

the entire raw data is an xml and to keep the format of the masked field i need OrbitalConnectionUsername also as tag as shown above.

Is there any other way i can send the raw data to you , if this is not clear ?

Re: How can i mask field values of AccountNum,StartAccountNum and EUDDIBAN that are extracted from xml tags . props.conf for reference

Sujithkumarkb — Tue, 27 Aug 2019 06:42:50 GMT

Hi @DavidHourani
Any suggestions on this ?

Presently i am getting data populated as below , which is not in between tags.

$AccountNum=########
OrderIDT1 18609,875D5F7D7D07939FDBBF37179FD7AC20474166545B20100B /OrderID
TxRefNum 5D5F7D7D07939FDBBF37179FD7AC20474166545B /TxRefNum

Re: How can i mask field values of AccountNum,StartAccountNum and EUDDIBAN that are extracted from xml tags . props.conf for reference

DavidHourani — Tue, 27 Aug 2019 08:42:14 GMT

Hi @Sujithkumarkb,

The $1 and $2 in the FORMAT line represent the capturing groups from the regex. If the capturing groups don't include the tags then they won't show in the format.

Try this as a REGEX:

REGEX = (.*\<OrbitalConnectionUsername\>).*(\<OrbitalConnectionUsername\>.*)

And this as a FORMAT:

FORMAT = $1########$2

Let me know if that helps.

Re: How can i mask field values of AccountNum,StartAccountNum and EUDDIBAN that are extracted from xml tags . props.conf for reference

Sujithkumarkb — Tue, 27 Aug 2019 10:14:26 GMT

@DavidHourani doesnt work ... 😞

The field extraction isnt happening nor the masking with the suggested regex

Re: How can i mask field values of AccountNum,StartAccountNum and EUDDIBAN that are extracted from xml tags . props.conf for reference

DavidHourani — Tue, 27 Aug 2019 10:17:05 GMT

try the regex like this :

(.*<OrbitalConnectionUsername>).*(<OrbitalConnectionUsername>.*)

Please post a sample data wrapped in "code sample" from splunk answers so I can use it to test out the regex for you.

Also apply the regex above on a sample of your data on regex101.com to make sure it's matching.

Re: How can i mask field values of AccountNum,StartAccountNum and EUDDIBAN that are extracted from xml tags . props.conf for reference

Sujithkumarkb — Tue, 27 Aug 2019 10:30:44 GMT

i checked both the regex and few of mine as well , extraction isnt happening
Below xml for refernce

D Tue Jul 30 11:39:01 2019 /opt/app/BRM/workspace

<?xml version="1.0" encoding="UTF-8"?>
<Response>
<NewOrderResp>
<MerchantID>111</MerchantID>
<TerminalID>001</TerminalID>
<AccountNum>XXXXXXXXXXXX7133</AccountNum>
<OrderID>T1,1822d,4</OrderID>
<OrbitalConnectionUsername>ABC123</OrbitalConnectionUsername>       
<OrbitalConnectionPassword>ZXY567</OrbitalConnectionPassword>
</NewOrderResp></Response>

Re: How can i mask field values of AccountNum,StartAccountNum and EUDDIBAN that are extracted from xml tags . props.conf for reference

DavidHourani — Tue, 27 Aug 2019 10:44:34 GMT

There seems to be a backslash missing in the REGEX, try it like this :

(?m)^(.*<OrbitalConnectionUsername>).*(<\/OrbitalConnectionUsername>.*)$

You can view it here :
https://regex101.com/r/SwIXOs/1

Re: How can i mask field values of AccountNum,StartAccountNum and EUDDIBAN that are extracted from xml tags . props.conf for reference

DavidHourani — Tue, 27 Aug 2019 10:53:23 GMT

If the above doesn't working for you then instead of using props and transforms and use sedcmd as I mentioned initially in the answer.

It should look like this in props.conf -- already tested it and it's working :

SEDCMD-Anon= s/<OrbitalConnectionUsername>([^<])+/<OrbitalConnectionUsername>XXXXXXXXXX/g

Re: How can i mask field values of AccountNum,StartAccountNum and EUDDIBAN that are extracted from xml tags . props.conf for reference

Sujithkumarkb — Tue, 27 Aug 2019 11:07:35 GMT

Thanks soo much David, both the solutions works perfect for me .
A Backslash played me pretty well.

Cheers 🙂

Re: How can i mask field values of AccountNum,StartAccountNum and EUDDIBAN that are extracted from xml tags . props.conf for reference

DavidHourani — Tue, 27 Aug 2019 11:21:44 GMT

hahah... yeah its annoying when this happens, you end up spending so much time troubleshooting something small like a backslash 🙂

Glad I could help, would really appreciate it if you can up-vote the answer and the comments, thank you !