Security

Data Masking Before Ingestion

anandhalagaras1
Contributor

Hi Team,

Want to mask two of the fields "password" and "cpassword" from the events which are getting written with the plain text information. So needs to be changed as #####.

Sample event information:

 

[2024-01-31_07:58:28] INFO : REQUEST: User:abc CreateUser POST: name: AB_Test_Max;email: xyz@gmail.com;password: abc12345679;cpassword: abc12345679;role: User;

[2024-01-30_14:05:42] INFO : REQUEST: User:xyz CreateUser POST: name: Math_Lab;email: abc@yahoo.com;password: xyzab54;cpassword: xyzab54;role: Admin;

So kindly help with the props.conf so that i can apply with SEDCMD-mask.

Labels (2)
0 Karma
1 Solution

marnall
Motivator

Could you try this SEDCMD in the props.conf file? (Make sure that the stanza is changed to match the sourcetype of the logs)

[your_sourcetype]
SEDCMD-maskpasswords = s/password: ([^;]+);cpassword: ([^;]+);/password: ####;cpassword: ####;/g

 

View solution in original post

marnall
Motivator

Could you try this SEDCMD in the props.conf file? (Make sure that the stanza is changed to match the sourcetype of the logs)

[your_sourcetype]
SEDCMD-maskpasswords = s/password: ([^;]+);cpassword: ([^;]+);/password: ####;cpassword: ####;/g

 

gcusello
SplunkTrust
SplunkTrust

Hi @anandhalagaras1,please try this:

[your_sourcetype]
SEDCMD = s/password: ([^;]+);cpassword: ([^;]+);/password: (####);cpassword: (####);/gm

that you can test at https://regex101.com/r/ppaFZc/1

Ciao.

Giuseppe

anandhalagaras1
Contributor

@gcusello 

We had two requirements for the same sourcetype. One involved line breaks, and the other required password masking during ingestion. As our Search heads are managed by Splunk Support and hosted in the Cloud, we created a custom app and deployed the props.conf in the default directory. After uploading the apps for the cloud vetting process, they were successfully installed. However, I've noticed that the logs are now being separated into individual events, which is acceptable, but the passwords are still visible and haven't been masked according to our requirement. I'm unsure about where exactly I may have missed it.

 

This is the props.conf file for reference. 

[sourcetype]
SHOULD_LINEMERGE = false
SEDCMD = s/password: ([^;]+);cpassword: ([^;]+);/password: (####);cpassword: (####);/gm

 

Sample log for reference: 

[2024-03-01_06:32:08] INFO : REQUEST: User:abc CreateUser POST: name: xyz;email: abc@gmail.com;password: xyz@123;cpassword: xyz@123;role: Administrator;

So kindly help on this requirement.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @anandhalagaras1,

regex substitution is correct.

Are you sure about the sourcetype?

is there any sourcetype replacement in your data?

are there some other Heavy Forwarders  before the one you used for the masking?

Ciao.

Giuseppe

0 Karma

anandhalagaras1
Contributor

@gcusello ,

This is the exact and correct sourcetype and I have created a custom app and uploaded the App in our Search head. Since our Search head is hosted in Splunk Cloud managed by Support.

So I have uploaded the app in the upload app section and post vetting process completed i have installed the custom app into the Search head.

This is the custom app i have created "abc_app"

Under abc_app I have placed two folders "default" and "metadata"

Under default I have created the app.conf and props.conf

And under metadata I have created the default.metadata 

Refer screenshots for reference.

 

So kindly let me know where i am missing since the lines are getting segregated as separate events whereas password masking is not getting applied to the events. Hence kindly help on the same.

 

 

 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @anandhalagaras1,

what's the sourcetype to apply the masking?

I suppose that sourcetype in the props.conf stanza header it's only for example and that in your installation you have the correct sourcetype to apply the transformation.

ciao.

Giuseppe

0 Karma

anandhalagaras1
Contributor

@gcusello  Indeed, I have applied the correct sourcetype there to ensure that events are appropriately divided. Nonetheless, the masking of passwords is not taking place as intended.

0 Karma

anandhalagaras1
Contributor

@gcusello ,

Any inputs from your end since still i can see the events are getting ingested with the password information present in it.

 

 

0 Karma
Get Updates on the Splunk Community!

New Dates, New City: Save the Date for .conf25!

Wake up, babe! New .conf25 dates AND location just dropped!! That's right, this year, .conf25 is taking place ...

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...