Not able to mask JSON before/at the time of indexing

[meenal901]

Hi,

I have json data to be parsed and there is a field "password" which needs to be masked at index time. Following the wiki page https://wiki.splunk.com/Community:How_to_mask_strings_in_json_event_at_Indexing_time_when_using_INDE... I am able to mask the _raw but the field still shows plain text.

Here's is my props and transforms and screenshot from search head:

** props.conf **
test_json_passwordINDEXED_EXTRACTIONS = json
TIMESTAMP_FIELDS = @timestamp
SEDCMD-mask_password_raw = s/\S+( - password)/"######\1/
TRANSFORMS-mask_json_password = mask_json_password_meta
KV_MODE = none
category = Structured
description = JavaScript Object Notation format. For more information, visit http://json.org/
disabled = false
pulldown_type = true

** transforms.conf **
[mask_json_password_meta]
SOURCE_KEY = _meta
DEST_KEY = _meta
REGEX = ^(.message[tT]ext::)\S+ - password" (.)
FORMAT = $1"###### - password" $2
WRITE_META = false

** Sample Data **
{
"username" : "my_username",
"password" : "my_password - password",
"validation-factors" : {
"validationFactors" : [
{
"name" : "remote_address",
"value" : "127.0.0.1"
}
]
},
"@timestamp": "2018-01-05T14:56:29.000Z",
"attributes": {
"field_1": "value_1",
"field_2": "value_2",
"field_3": "value_3"
}
}
1) Masked data in raw event

2) Clear password visible in password field

[cpetterborg]

You can't do it on the search head. Your props.conf and transforms.conf must be on the indexers for this to work. It should be possible to do this with only the props file.

[FrankVl]

Given that the SEDCMD works, the config must be on his indexers already.

Could it be that the "indexed extractions" take place before the SEDCMD? Solution may be to define a specific TRANSFORMS with the password field as SOURCE_KEY and DEST_KEY and then apply the masking again.

Edit: Oh, and shoot the person who thought it was a good idea to write plaintext passwords to a log in the first place 😛

[meenal901]

Oh I wish I could 😛

I will try the SOURCE_KEY and DEST_KEY as password and try again.