Getting Data In

Sanitize Incoming Data (remove Passwords)

aattinello
Engager

Hello,
I am looking for a sanitize my incoming data. My customers sometimes pass GET parameters instead of POST parameters, which is normally fine. However in some cases they pass their password into our API as a GET parameter which then appears in plain text in my webserver log. When i send this data to Splunk I would like to match that password and replace with a string like FILTERED. The tool I am using to log this data has no way to scrub that data while preserving the other get parameters, so I was hoping Splunk was able to.

So some of my sample lines looks like this
10.213.172.3 [02/May/2014:16:31:07 -0400] 31249 "GET /endPoint/?action=login&loginUsername=test&loginOrganization=Test&loginPassword=superTest HTTP/1.1" 200 570 4243 "Zend_Http_Client" "-" -

10.213.172.3 [02/May/2014:16:31:16 -0400] 187498 "POST /endpoint/other/otherPage.html?loginUsername=test&loginPassword=superTest&loginOrganization=Test HTTP/1.1" 200 1573 708 "Zend_Http_Client" "en-US,en;q=0.8" 6E1182505E7B71DAA4340E831A53F440.node1

I am looking to match this parameter (up until the first space or &)
&loginPassword=((.*&)|(\S+))
And replace that with something like
&loginPassword=FILTERED

So those 2 examples would end up indexed as
10.213.172.3 [02/May/2014:16:31:07 -0400] 31249 "GET /endPoint/?action=login&loginUsername=test&loginOrganization=Test&loginPassword=FILTERED HTTP/1.1" 200 570 4243 "Zend_Http_Client" "-" -

10.213.172.3 [02/May/2014:16:31:16 -0400] 187498 "POST /endpoint/other/otherPage.html?loginUsername=test&loginPassword=FILTERED&loginOrganization=Test HTTP/1.1" 200 1573 708 "Zend_Http_Client" "en-US,en;q=0.8" 6E1182505E7B71DAA4340E831A53F440.node1

0 Karma
1 Solution

gfreitas
Builder

Hi aattinello,

I know you can mask sensitive data using props.conf and transforms.conf.

In props.conf:

[source::\\yoursource.log]
TRANSFORMS-password = password_mask

And in transforms.conf:

[password_mask]
DEST_KEY = _raw
REGEX = (.*loginPassword=)\d\s
FORMAT = $FILTERED$

I don't understand a lot of regex, but maybe you can modify it using some online checker.

Hope this helps!

View solution in original post

gfreitas
Builder

Hi aattinello,

I know you can mask sensitive data using props.conf and transforms.conf.

In props.conf:

[source::\\yoursource.log]
TRANSFORMS-password = password_mask

And in transforms.conf:

[password_mask]
DEST_KEY = _raw
REGEX = (.*loginPassword=)\d\s
FORMAT = $FILTERED$

I don't understand a lot of regex, but maybe you can modify it using some online checker.

Hope this helps!

aattinello
Engager

Yes, that is what i was looking for, thank you very much.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...