Sanitize Incoming Data (remove Passwords)

[aattinello]

Hello,
I am looking for a sanitize my incoming data. My customers sometimes pass GET parameters instead of POST parameters, which is normally fine. However in some cases they pass their password into our API as a GET parameter which then appears in plain text in my webserver log. When i send this data to Splunk I would like to match that password and replace with a string like FILTERED. The tool I am using to log this data has no way to scrub that data while preserving the other get parameters, so I was hoping Splunk was able to.

So some of my sample lines looks like this
10.213.172.3 [02/May/2014:16:31:07 -0400] 31249 "GET /endPoint/?action=login&loginUsername=test&loginOrganization=Test&loginPassword=superTest HTTP/1.1" 200 570 4243 "Zend_Http_Client" "-" -

10.213.172.3 [02/May/2014:16:31:16 -0400] 187498 "POST /endpoint/other/otherPage.html?loginUsername=test&loginPassword=superTest&loginOrganization=Test HTTP/1.1" 200 1573 708 "Zend_Http_Client" "en-US,en;q=0.8" 6E1182505E7B71DAA4340E831A53F440.node1

I am looking to match this parameter (up until the first space or &)
&loginPassword=((.*&)|(\S+))
And replace that with something like
&loginPassword=FILTERED

So those 2 examples would end up indexed as
10.213.172.3 [02/May/2014:16:31:07 -0400] 31249 "GET /endPoint/?action=login&loginUsername=test&loginOrganization=Test&loginPassword=FILTERED HTTP/1.1" 200 570 4243 "Zend_Http_Client" "-" -

10.213.172.3 [02/May/2014:16:31:16 -0400] 187498 "POST /endpoint/other/otherPage.html?loginUsername=test&loginPassword=FILTERED&loginOrganization=Test HTTP/1.1" 200 1573 708 "Zend_Http_Client" "en-US,en;q=0.8" 6E1182505E7B71DAA4340E831A53F440.node1

[gfreitas · Accepted solution]

Hi aattinello,

I know you can mask sensitive data using props.conf and transforms.conf.

In props.conf:

[source::\\yoursource.log]
TRANSFORMS-password = password_mask

And in transforms.conf:

[password_mask]
DEST_KEY = _raw
REGEX = (.*loginPassword=)\d\s
FORMAT = $FILTERED$

I don't understand a lot of regex, but maybe you can modify it using some online checker.

Hope this helps!

[aattinello]

Yes, that is what i was looking for, thank you very much.