Getting Data In

Not able to mask JSON before/at the time of indexing



I have json data to be parsed and there is a field "password" which needs to be masked at index time. Following the wiki page I am able to mask the _raw but the field still shows plain text.

Here's is my props and transforms and screenshot from search head:

** props.conf **
test_json_passwordINDEXED_EXTRACTIONS = json
SEDCMD-mask_password_raw = s/\S+( - password)/"######\1/
TRANSFORMS-mask_json_password = mask_json_password_meta
KV_MODE = none
category = Structured
description = JavaScript Object Notation format. For more information, visit
disabled = false
pulldown_type = true

** transforms.conf **
SOURCE_KEY = _meta
DEST_KEY = _meta
REGEX = ^(.message[tT]ext::)\S+ - password" (.)
FORMAT = $1"###### - password" $2
WRITE_META = false

** Sample Data **
"username" : "my_username",
"password" : "my_password - password",
"validation-factors" : {
"validationFactors" : [
"name" : "remote_address",
"value" : ""
"@timestamp": "2018-01-05T14:56:29.000Z",
"attributes": {
"field_1": "value_1",
"field_2": "value_2",
"field_3": "value_3"
1) Masked data in raw event
alt text
2) Clear password visible in password field
alt text

0 Karma


You can't do it on the search head. Your props.conf and transforms.conf must be on the indexers for this to work. It should be possible to do this with only the props file.

0 Karma

Ultra Champion

Given that the SEDCMD works, the config must be on his indexers already.

Could it be that the "indexed extractions" take place before the SEDCMD? Solution may be to define a specific TRANSFORMS with the password field as SOURCE_KEY and DEST_KEY and then apply the masking again.

Edit: Oh, and shoot the person who thought it was a good idea to write plaintext passwords to a log in the first place 😛

0 Karma


Oh I wish I could 😛

I will try the SOURCE_KEY and DEST_KEY as password and try again.

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!