I have a systemout.log file and I am indexing using pretrained sourcetype websphere_trlog_sysout. Currently there is an issue with masking
I have created props.conf in the deployment app and deployed to universal forwarder as below. Created transforms.conf to mask the data.
Seems the issue is still same.
[websphere_trlog_sysout] TIME_FORMAT = %d-%m-%y %H:%M:%S:%3Q %Z TIME_PREFIX = ^\[ FORMAT = $1-$2-$3 $4:$5:$6:$7 $8 TRANSFORMS-anonymize = session-anonymizer [session-anonymizer] REGEX = XXXXX FORMAT = $1XXXX$2 DEST_KEY = _raw
Thanks for the reply I am trying to mask card details, that does not have field
Ex: [23/03/2020 13:45:20:123 IST Systemout ..... Card 1234-5678-9012-3456]
I would like to mask card details.
try the below search
|makeresults |eval data="23/03/2020 13:45:20:123 IST Systemout ..... Card 1234-5678-9012-3456"|rex mode=sed field=data "s/Card ((\d+\-)+)/Card XXXX-XXXX-XXXX/"
and try it in props.conf
SEDCMD-replace=s/Card ((\d+\-)+)/Card XXXX-XXXX-XXXX/
It is resolved the issue partially. I found Card details in other location as well
As sed command have limitation to use only once, I would like to use regex.
I tried regex command as stated above in the custom app deployed in forwarder. But this is not working at all.
where do you think I did the mistake?
TRANSFORMS-anonymize = session-anonymizer
REGEX = XXXXX
FORMAT = $1XXXX$2
DEST_KEY = _raw
Hey not sure how you came to conclusion that As "sed command have limitation to use only once" there is no such limitation it would replace multiple occurences of the matched regex if the pattern is same you just have to add global attribute "g" at the end
|rex mode=sed field=data "s/Card ((\d+\-)+)/Card XXXX-XXXX-XXXX/g"
|makeresults |eval data="23/03/2020 13:45:20:123 IST Systemout ..... Card 1234-5678-9012-3456 Card 1234-5678-9012-3456" |rex mode=sed field=data "s/Card ((\d+\-)+)/Card XXXX-XXXX-XXXX/g"
Note this would only not work in case the pattern is not matching for the second occcurence
And about your Transforms.conf
REGEX = XXXX ( you have to define the regex which would be the identifier )
FORMAT = $1XXXX$2 ($1 and $2) are the captured named group
DEST_KEY = _raw
REGEX = (Card) ((\d+\-)+) ( this is the regex)
FORMAT = $1XXXX (here there wont be $2 because there is only one group here and your are masking 2nd group so dont include $2
DEST_KEY = _raw
if you still have trouble i would share more details with screenshot
If you find the answer was helpful, an upvote/karma is appreciated and please accept as solution it would help others as well
you can try the sedcmd in props.conf and dont require in transform but this should go to indexer
SED script works at index time and executed on _raw field. so should be in indexer
first you can test the sedcmd in a rex in a search: to check if masking is working
|makeresults |eval date="2022-03-23 10:24:19:695+0000"|rex mode=sed field=date max_match=0 "s/(([\d+\-]+) ([\d+:]+))/XXXXXX/"
Just write this is in props.conf . you do not need to write transforms.conf.
and then restart the server.
Hope this helps