Splunk Search

How to write a regex to extract a field within a particular field?

sundarrajan
Path Finder

I am looking for a way to some how extract and mask some of important information that comes within logs. I don't have absolute permission to access "props.conf" and "transforms.conf" and hence i have to first search for the key word, extract the key field and then then mask it. The information that i wish to mask is password and login credentials which is part of field .
USER_CREDENTIAL, field is of the format \"userID\":\"ABC1234\" ,\"password\":\"abcd345\", \"email\":\"abcd@gmail.com\",\"country\":\"AAAA\"
i tried the following format to extract "password", rex field=USER_CREDENTIAL "(?\w+\D+\w+[^\]) but it extract both userID and password as PASSWORD,
I am also looking for an easy way to mask the same, earlier i tried of masking the details but i was left with masking both userID and password, basesearch | rex field=USER_CREDENTIAL mode=sed "s/(\w+)(\D+)(\w+)(\w+)/1\\":\\"XXXXXXX/2", but the masking is not efficient when the format in userID changes. Hence thought of extracting the field and then masking the same.
Please do suggest if there could be any easy way to get the masking done.

0 Karma
1 Solution

somesoni2
Revered Legend

Give this a try. First two lines are to generate sample data, replace that with your search. First rex extracts fields and second rex masks the sensitive information

| gentimes start=-1 | eval USER_CREDETIAL="\\\"userID\\\":\"ABC1234\\\" ,\\\"password\\\":\\\"abcd345\\\", \\\"email\\\":\\\"abcd@gmail.com\\\",\\\"country\\\":\\\"AAAA\\\"" | table USER_CREDETIAL 
| rex field=USER_CREDETIAL "userID([^\"]+\"){2}(?<userID>[^\\\]+).+password([^\"]+\"){2}(?<password>[^\\\]+)" 
| rex field=USER_CREDETIAL mode=sed "s/(userID[^\"]+\"[^\"]+\")([^\\\]+)(.+password[^\"]+\"[^\"]+\")([^\\\]+)/\1XXXXX\3XXXX/"

View solution in original post

0 Karma

somesoni2
Revered Legend

Give this a try. First two lines are to generate sample data, replace that with your search. First rex extracts fields and second rex masks the sensitive information

| gentimes start=-1 | eval USER_CREDETIAL="\\\"userID\\\":\"ABC1234\\\" ,\\\"password\\\":\\\"abcd345\\\", \\\"email\\\":\\\"abcd@gmail.com\\\",\\\"country\\\":\\\"AAAA\\\"" | table USER_CREDETIAL 
| rex field=USER_CREDETIAL "userID([^\"]+\"){2}(?<userID>[^\\\]+).+password([^\"]+\"){2}(?<password>[^\\\]+)" 
| rex field=USER_CREDETIAL mode=sed "s/(userID[^\"]+\"[^\"]+\")([^\\\]+)(.+password[^\"]+\"[^\"]+\")([^\\\]+)/\1XXXXX\3XXXX/"
0 Karma

yannK
Splunk Employee
Splunk Employee

Assuming your event is like :

  \"userID\":\"ABC1234\" ,\"password\":\"abcd345\"

For the sed replacement, have you tried to use a broader condition, by example the double quote character as string limit for your values ?

mysearch | rex mod=sed "s/\"userID\":\"([^\"]*)\"/\"userID\":\"XXXX\"/g"

mysearch | rex mod=sed "s/\"password\":\"([^\"]*)\"/\"password\":\"XXXX\"/g"

It will of course fail if your user or password do contains doublequotes in it.

then you can merge in one sed, or use 2 sed commands.

0 Karma
Get Updates on the Splunk Community!

SplunkTrust Application Period is Officially OPEN!

It's that time, folks! The application/nomination period for the 2025 SplunkTrust is officially open! If you ...

Splunk Answers Content Calendar, June Edition II

Get ready to dive into Splunk Dashboard panels this week! We'll be tackling common questions around ...

Splunk Observability Cloud's AI Assistant in Action Series: Auditing Compliance and ...

This is the third post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...