How to get timestamp and mask syslog through universal forwarder?

[marellasunil_3]

I have a systemout.log file and I am indexing using pretrained sourcetype websphere_trlog_sysout. Currently there is an issue with masking

I have created props.conf in the deployment app and deployed to universal forwarder as below. Created transforms.conf to mask the data.

Seems the issue is still same. 

[websphere_trlog_sysout]
TIME_FORMAT = %d-%m-%y %H:%M:%S:%3Q %Z
TIME_PREFIX = ^\[
FORMAT = $1-$2-$3 $4:$5:$6:$7 $8
TRANSFORMS-anonymize = session-anonymizer

[session-anonymizer]

REGEX = XXXXX

FORMAT = $1XXXX$2

DEST_KEY = _raw

[marellasunil_3]

Hi Venky
Thanks for the reply I am trying to mask card details, that does not have field

Ex: [23/03/2020 13:45:20:123 IST Systemout .....      Card 1234-5678-9012-3456]

I would like to mask card details.

[venky1544]

Hi @marellasunil_3 

try the below search 

|makeresults |eval data="23/03/2020 13:45:20:123 IST Systemout ..... Card 1234-5678-9012-3456"|rex mode=sed field=data "s/Card ((\d+\-)+)/Card XXXX-XXXX-XXXX/"

and try it in props.conf 

[websphere_trlog_sysout]

SEDCMD-replace=s/Card ((\d+\-)+)/Card XXXX-XXXX-XXXX/

---
If this reply helps you, an upvote/karma  would be appreciated.

[marellasunil_3]

Thanks Venky.

It is resolved the issue partially. I found Card details in other location as well 

As sed command have limitation to use only once, I would like to use regex.

I tried regex command as stated above in the custom app deployed in forwarder. But this is not working at all.

where do you think I did the mistake? 

Props.comf

[websphere_trlog_sysout]
TRANSFORMS-anonymize = session-anonymizer

Transforms.conf

[session-anonymizer]
REGEX = XXXXX
FORMAT = $1XXXX$2
DEST_KEY = _raw

[venky1544]

Hi @marellasunil_3 

Hey not sure how you came to conclusion that As "sed command have limitation to use only once" there is no such limitation it would replace multiple occurences of the matched regex if the pattern is same you just have to add global attribute "g" at the end

|rex mode=sed field=data "s/Card ((\d+\-)+)/Card XXXX-XXXX-XXXX/g"

example 

|makeresults |eval data="23/03/2020 13:45:20:123 IST Systemout ..... Card 1234-5678-9012-3456 Card 1234-5678-9012-3456" |rex mode=sed field=data "s/Card ((\d+\-)+)/Card XXXX-XXXX-XXXX/g"

Note this would only not work in case the pattern is not matching for the second occcurence 

And about your Transforms.conf 

[session-anonymizer]
REGEX = XXXX ( you have to define the regex which would be the identifier )
FORMAT = $1XXXX$2 ($1 and $2) are the captured named group
DEST_KEY = _raw

[session-anonymizer]
REGEX = (Card) ((\d+\-)+) ( this is the regex)
FORMAT = $1XXXX (here there wont be $2 because there is only one group here and your are masking 2nd group so dont include $2
DEST_KEY = _raw

if you still have trouble i would share more details with screenshot

--------

If you find the answer was helpful, an upvote/karma is appreciated and please accept as solution it would help others as well

[venky1544]

Hi @marellasunil_3 

you can try the sedcmd in props.conf and dont require in transform but this should go to indexer
SED script works at index time and executed on _raw field. so should be in indexer

first you can test the  sedcmd in a rex in a search: to check if masking is working 

|makeresults |eval date="2022-03-23 10:24:19:695+0000"|rex mode=sed field=date max_match=0 "s/(([\d+\-]+) ([\d+:]+))/XXXXXX/"

Just write this is in props.conf . you do not need to write transforms.conf.

[websphere_trlog_sysout]

SEDCMD-replace=/(([\d+\-]+) ([\d+:]+))/XXXXXX/

and then restart the server.

Hope this helps