Hi,
Am trying to do an index time masking where my data is not in _raw but in a separate field A.
For example A field has the following data
"Path=/LoginUser Query=CrmId=ClientABC& ContentItemId=TotalAccess&SessionId=3A1785URH117BEA&Ticket=646A1DA4STF896EE& SessionTime=25368&ReturnUrl=http://www.clientabc.com, Method=GET,IP=209.51.249.195, Content=", ""
I have applied transforms rules as below,
[session-anonymizer]
SOURCE_KEY = field:A REGEX = (?m)^(.*)SessionId=\w+(\w{4}[&"].*)$ FORMAT = $1SessionId=########$2 DEST_KEY = field:A
The problem is when we give the DEST_KEY as _raw it is masked properly, But I need the masked data back to field A. How do we get this masked to field:A
I have also tried adding
[accepted_keys]
is_valid = field:A
Have you tried masking using SEDCMD? It's simpler than using transforms. Put this in props.conf:
[mysourcetype]
SEDCMD-maskSessionID = s/SessionId=[^&]+/SessionId=########/g
Thanks for looking into this.
the problem is my data is not in _raw but in field A.
it comes as an additional field from hec as indexed field.
so SEDCMD has its limitation of being applied directly on _raw and not on indexed field.
thats why I had to use the SOURCE_KEY = field:A
INGEST_EVAL with replace solved the issue