Getting Data In

Not able to mask JSON before/at the time of indexing

meenal901
Communicator

Hi,

I have json data to be parsed and there is a field "password" which needs to be masked at index time. Following the wiki page https://wiki.splunk.com/Community:How_to_mask_strings_in_json_event_at_Indexing_time_when_using_INDE... I am able to mask the _raw but the field still shows plain text.

Here's is my props and transforms and screenshot from search head:

** props.conf **
test_json_passwordINDEXED_EXTRACTIONS = json
TIMESTAMP_FIELDS = @timestamp
SEDCMD-mask_password_raw = s/\S+( - password)/"######\1/
TRANSFORMS-mask_json_password = mask_json_password_meta
KV_MODE = none
category = Structured
description = JavaScript Object Notation format. For more information, visit http://json.org/
disabled = false
pulldown_type = true

** transforms.conf **
[mask_json_password_meta]
SOURCE_KEY = _meta
DEST_KEY = _meta
REGEX = ^(.message[tT]ext::)\S+ - password" (.)
FORMAT = $1"###### - password" $2
WRITE_META = false

** Sample Data **
{
"username" : "my_username",
"password" : "my_password - password",
"validation-factors" : {
"validationFactors" : [
{
"name" : "remote_address",
"value" : "127.0.0.1"
}
]
},
"@timestamp": "2018-01-05T14:56:29.000Z",
"attributes": {
"field_1": "value_1",
"field_2": "value_2",
"field_3": "value_3"
}
}
1) Masked data in raw event
alt text
2) Clear password visible in password field
alt text

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

You can't do it on the search head. Your props.conf and transforms.conf must be on the indexers for this to work. It should be possible to do this with only the props file.

0 Karma

FrankVl
Ultra Champion

Given that the SEDCMD works, the config must be on his indexers already.

Could it be that the "indexed extractions" take place before the SEDCMD? Solution may be to define a specific TRANSFORMS with the password field as SOURCE_KEY and DEST_KEY and then apply the masking again.

Edit: Oh, and shoot the person who thought it was a good idea to write plaintext passwords to a log in the first place 😛

0 Karma

meenal901
Communicator

Oh I wish I could 😛

I will try the SOURCE_KEY and DEST_KEY as password and try again.

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!