Hello,
Since i am new to Splunk, i'm having hard time understanding and writing the transforms for varying password lengths. It would be great if someone could help me with transforms to mask the username and password. As you may have noticed the password is not of fixed length–it varies for each event.
SAMPLE DATA:
192.56.052.10 32.200.12.677, 123.23.234.31 - - [08/Feb/2016:16:41:44 -0500] "GET /login?company=monitorabc34adte1&bplte=&username=admin&password=welcome234 HTTP/1.1" 505 12345 "-" "alexa.com_bot_version_1.4_(http://www.alexa.com/)" "-" "asjenkwqjdqcqwjdncqkwddwc" "-" 0
192.56.052.10 32.200.12.677, 123.23.234.31 - - [08/Feb/2016:16:41:44 -0500] "GET /login?company=D0110&bplte=&username=grante&password=abc123 HTTP/1.1" 505 12345 "-" "alexa.com_bot_version_1.4_(http://www.alexa.com/)" "-" "asjenkwqjdqcqwjdncqkwddwc" "-" 0
192.56.052.10 32.200.12.677, 123.23.234.312 - - [08/Feb/2016:16:41:44 -0500] "GET /login?company=1234567u&username=asqwdcfVPOE$rV&password=ssqwerfV78#deQA&tklogin_key=abcdefg123 HTTP/1.1" 302 - "https://www.google.com/URLredirected/abc" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) mywebkit/537.36 (KHTML, like appesi) Chrome/62.0.3202.94 Safari/537.36" "-" "asdqwdqwdsfdewwichedf" "-" 0
Hi @kteng2024,
You can try below props.conf and transforms.conf
props.conf
[yoursourcetype]
TRANSFORMS-anonymize = anopassword, anousername
transforms.conf
[anopassword]
REGEX = (?m)^(.*)(?=password\=)[^\s|&]*(.*)$
FORMAT = $1password=####$2
DEST_KEY = _raw
[anousername]
REGEX = (?m)^(.*)(?=username\=)[^&]*(.*)$
FORMAT = $1username=####$2
DEST_KEY = _raw
If you do not want use transforms.conf then you can achieve same using SEDCMD in props.conf
So you need to configure props.conf as below
[yoursourcetype]
SEDCMD-maskall = s/(?m)^(.*username\=)[^&]*(.*password\=)[^\s|&]*(.*)$/\1######\2######\3/g
Based on sample data I have created below sample query (First 4 lines used to generate dummy data) with sed regex which I have given above
| makeresults
| eval _raw="192.56.052.10 32.200.12.677, 123.23.234.31 - - [08/Feb/2016:16:41:44 -0500] \"GET /login?company=monitorabc34adte1&bplte=&username=admin&password=welcome234 HTTP/1.1\" 505 12345 \"-\" \"alexa.com_bot_version_1.4_(http://www.alexa.com/)\" \"-\" \"asjenkwqjdqcqwjdncqkwddwc\" \"-\" 0"
| append [ | makeresults| eval _raw="192.56.052.10 32.200.12.677, 123.23.234.31 - - [08/Feb/2016:16:41:44 -0500] \"GET /login?company=D0110&bplte=&username=grante&password=abc123 HTTP/1.1\" 505 12345 \"-\" \"alexa.com_bot_version_1.4_(http://www.alexa.com/)\" \"-\" \"asjenkwqjdqcqwjdncqkwddwc\" \"-\" 0"]
| append [ | makeresults| eval _raw="192.56.052.10 32.200.12.677, 123.23.234.312 - - [08/Feb/2016:16:41:44 -0500] \"GET /login?company=1234567u&username=asqwdcfVPOE$rV&password=ssqwerfV78#deQA&tklogin_key=abcdefg123 HTTP/1.1\" 302 - \"https://www.google.com/URLredirected/abc\" \"Mozilla/5.0 (Windows NT 6.1; Win64; x64) mywebkit/537.36 (KHTML, like appesi) Chrome/62.0.3202.94 Safari/537.36\" \"-\" \"asdqwdqwdsfdewwichedf\" \"-\" 0" ]
| rex mode=sed "s/(?m)^(.*username\=)[^&]*(.*password\=)[^\s|&]*(.*)$/\1######\2######\3/g"
which is giving result as below
192.56.052.10 32.200.12.677, 123.23.234.31 - - [08/Feb/2016:16:41:44 -0500] "GET /login?company=monitorabc34adte1&bplte=&username=######&password=###### HTTP/1.1" 505 12345 "-" "alexa.com_bot_version_1.4_(http://www.alexa.com/)" "-" "asjenkwqjdqcqwjdncqkwddwc" "-" 0
192.56.052.10 32.200.12.677, 123.23.234.31 - - [08/Feb/2016:16:41:44 -0500] "GET /login?company=D0110&bplte=&username=######&password=###### HTTP/1.1" 505 12345 "-" "alexa.com_bot_version_1.4_(http://www.alexa.com/)" "-" "asjenkwqjdqcqwjdncqkwddwc" "-" 0
192.56.052.10 32.200.12.677, 123.23.234.312 - - [08/Feb/2016:16:41:44 -0500] "GET /login?company=1234567u&username=######&password=######&tklogin_key=abcdefg123 HTTP/1.1" 302 - "https://www.google.com/URLredirected/abc" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) mywebkit/537.36 (KHTML, like appesi) Chrome/62.0.3202.94 Safari/537.36" "-" "asdqwdqwdsfdewwichedf" "-" 0
I hope this helps.
Thanks,
Harshil
Using ONLY the data you provided, this is what you could put in props.conf
:
[host::*]
TRANSFORMS-anonymize1 = nousernameuri
TRANSFORMS-anonymize2 = nopassworduri
And transforms.conf
:
[nopassworduri]
REGEX = (?m)^([\s\S]*)password=[^& "]+([\s\S]*)$
FORMAT = $1password=#####$2
DEST_KEY = _raw
[nousernameuri]
REGEX = (?m)^([\s\S]*)username=[^& "]+([\s\S]*)$
FORMAT = $1username=#####$2
DEST_KEY = _raw
The username
and password
will be a fixed length #####
and not the same length as the original values.