How to write a regex to extract a field within a particular field?

[sundarrajan]

I am looking for a way to some how extract and mask some of important information that comes within logs. I don't have absolute permission to access "props.conf" and "transforms.conf" and hence i have to first search for the key word, extract the key field and then then mask it. The information that i wish to mask is password and login credentials which is part of field .
USER_CREDENTIAL, field is of the format \"userID\":\"ABC1234\" ,\"password\":\"abcd345\", \"email\":\"abcd@gmail.com\",\"country\":\"AAAA\"
i tried the following format to extract "password", rex field=USER_CREDENTIAL "(?\w+\D+\w+[^\]) but it extract both userID and password as PASSWORD,
I am also looking for an easy way to mask the same, earlier i tried of masking the details but i was left with masking both userID and password, basesearch | rex field=USER_CREDENTIAL mode=sed "s/(\w+)(\D+)(\w+)(\w+)/1\\":\\"XXXXXXX/2", but the masking is not efficient when the format in userID changes. Hence thought of extracting the field and then masking the same.
Please do suggest if there could be any easy way to get the masking done.

[somesoni2 · Accepted solution]

Give this a try. First two lines are to generate sample data, replace that with your search. First rex extracts fields and second rex masks the sensitive information

| gentimes start=-1 | eval USER_CREDETIAL="\\\"userID\\\":\"ABC1234\\\" ,\\\"password\\\":\\\"abcd345\\\", \\\"email\\\":\\\"abcd@gmail.com\\\",\\\"country\\\":\\\"AAAA\\\"" | table USER_CREDETIAL 
| rex field=USER_CREDETIAL "userID([^\"]+\"){2}(?<userID>[^\\\]+).+password([^\"]+\"){2}(?<password>[^\\\]+)" 
| rex field=USER_CREDETIAL mode=sed "s/(userID[^\"]+\"[^\"]+\")([^\\\]+)(.+password[^\"]+\"[^\"]+\")([^\\\]+)/\1XXXXX\3XXXX/"

[yannK]

Assuming your event is like :

  \"userID\":\"ABC1234\" ,\"password\":\"abcd345\"

For the sed replacement, have you tried to use a broader condition, by example the double quote character as string limit for your values ?

mysearch | rex mod=sed "s/\"userID\":\"([^\"]*)\"/\"userID\":\"XXXX\"/g"

mysearch | rex mod=sed "s/\"password\":\"([^\"]*)\"/\"password\":\"XXXX\"/g"

It will of course fail if your user or password do contains doublequotes in it.

then you can merge in one sed, or use 2 sed commands.