How to write the transforms.conf for varying password lengths

[kteng2024]

Hello,

Since i am new to Splunk, i'm having hard time understanding and writing the transforms for varying password lengths. It would be great if someone could help me with transforms to mask the username and password. As you may have noticed the password is not of fixed length–it varies for each event.

SAMPLE DATA:

192.56.052.10 32.200.12.677, 123.23.234.31 - - [08/Feb/2016:16:41:44 -0500] "GET /login?company=monitorabc34adte1&bplte=&username=admin&password=welcome234 HTTP/1.1" 505 12345 "-" "alexa.com_bot_version_1.4_(http://www.alexa.com/)" "-" "asjenkwqjdqcqwjdncqkwddwc" "-" 0

192.56.052.10 32.200.12.677, 123.23.234.31 - - [08/Feb/2016:16:41:44 -0500] "GET /login?company=D0110&bplte=&username=grante&password=abc123 HTTP/1.1" 505 12345 "-" "alexa.com_bot_version_1.4_(http://www.alexa.com/)" "-" "asjenkwqjdqcqwjdncqkwddwc" "-" 0

192.56.052.10 32.200.12.677, 123.23.234.312 - - [08/Feb/2016:16:41:44 -0500] "GET /login?company=1234567u&username=asqwdcfVPOE$rV&password=ssqwerfV78#deQA&tklogin_key=abcdefg123 HTTP/1.1" 302 - "https://www.google.com/URLredirected/abc" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) mywebkit/537.36 (KHTML, like appesi) Chrome/62.0.3202.94 Safari/537.36" "-" "asdqwdqwdsfdewwichedf" "-" 0

[harsmarvania57]

Hi @kteng2024,

You can try below props.conf and transforms.conf

props.conf

[yoursourcetype]
TRANSFORMS-anonymize = anopassword, anousername

transforms.conf

[anopassword]
REGEX = (?m)^(.*)(?=password\=)[^\s|&]*(.*)$
FORMAT = $1password=####$2
DEST_KEY = _raw

[anousername]
REGEX = (?m)^(.*)(?=username\=)[^&]*(.*)$
FORMAT = $1username=####$2
DEST_KEY = _raw

If you do not want use transforms.conf then you can achieve same using SEDCMD in props.conf

So you need to configure props.conf as below

[yoursourcetype]
SEDCMD-maskall = s/(?m)^(.*username\=)[^&]*(.*password\=)[^\s|&]*(.*)$/\1######\2######\3/g

Based on sample data I have created below sample query (First 4 lines used to generate dummy data) with sed regex which I have given above

| makeresults
| eval _raw="192.56.052.10 32.200.12.677, 123.23.234.31 - - [08/Feb/2016:16:41:44 -0500] \"GET /login?company=monitorabc34adte1&bplte=&username=admin&password=welcome234 HTTP/1.1\" 505 12345 \"-\" \"alexa.com_bot_version_1.4_(http://www.alexa.com/)\" \"-\" \"asjenkwqjdqcqwjdncqkwddwc\" \"-\" 0"
| append [ | makeresults| eval _raw="192.56.052.10 32.200.12.677, 123.23.234.31 - - [08/Feb/2016:16:41:44 -0500] \"GET /login?company=D0110&bplte=&username=grante&password=abc123 HTTP/1.1\" 505 12345 \"-\" \"alexa.com_bot_version_1.4_(http://www.alexa.com/)\" \"-\" \"asjenkwqjdqcqwjdncqkwddwc\" \"-\" 0"]
| append [ | makeresults| eval _raw="192.56.052.10 32.200.12.677, 123.23.234.312 - - [08/Feb/2016:16:41:44 -0500] \"GET /login?company=1234567u&username=asqwdcfVPOE$rV&password=ssqwerfV78#deQA&tklogin_key=abcdefg123 HTTP/1.1\" 302 - \"https://www.google.com/URLredirected/abc\" \"Mozilla/5.0 (Windows NT 6.1; Win64; x64) mywebkit/537.36 (KHTML, like appesi) Chrome/62.0.3202.94 Safari/537.36\" \"-\" \"asdqwdqwdsfdewwichedf\" \"-\" 0" ]
| rex mode=sed "s/(?m)^(.*username\=)[^&]*(.*password\=)[^\s|&]*(.*)$/\1######\2######\3/g"

which is giving result as below

192.56.052.10 32.200.12.677, 123.23.234.31 - - [08/Feb/2016:16:41:44 -0500] "GET /login?company=monitorabc34adte1&bplte=&username=######&password=###### HTTP/1.1" 505 12345 "-" "alexa.com_bot_version_1.4_(http://www.alexa.com/)" "-" "asjenkwqjdqcqwjdncqkwddwc" "-" 0
192.56.052.10 32.200.12.677, 123.23.234.31 - - [08/Feb/2016:16:41:44 -0500] "GET /login?company=D0110&bplte=&username=######&password=###### HTTP/1.1" 505 12345 "-" "alexa.com_bot_version_1.4_(http://www.alexa.com/)" "-" "asjenkwqjdqcqwjdncqkwddwc" "-" 0
192.56.052.10 32.200.12.677, 123.23.234.312 - - [08/Feb/2016:16:41:44 -0500] "GET /login?company=1234567u&username=######&password=######&tklogin_key=abcdefg123 HTTP/1.1" 302 - "https://www.google.com/URLredirected/abc" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) mywebkit/537.36 (KHTML, like appesi) Chrome/62.0.3202.94 Safari/537.36" "-" "asdqwdqwdsfdewwichedf" "-" 0

I hope this helps.

Thanks,
Harshil

[cpetterborg]

Using ONLY the data you provided, this is what you could put in props.conf:

[host::*]
TRANSFORMS-anonymize1 = nousernameuri
TRANSFORMS-anonymize2 = nopassworduri

And transforms.conf:

[nopassworduri]
REGEX = (?m)^([\s\S]*)password=[^& "]+([\s\S]*)$
FORMAT = $1password=#####$2
DEST_KEY = _raw

[nousernameuri]
REGEX = (?m)^([\s\S]*)username=[^& "]+([\s\S]*)$
FORMAT = $1username=#####$2
DEST_KEY = _raw

The username and password will be a fixed length ##### and not the same length as the original values.