topic Re: Source Transform Replace '/' with '_' in Getting Data In

Source Transform Replace '/' with '_'

ekremikizoglu — Tue, 29 Sep 2020 12:13:20 GMT

Hi,

I created props and transforms files to put source value of file in raw event. I am sending these event to third party app. I am using heavy forwarder. But ı need to replace "/",":"(non-alphanumeric) with "_" . Is there any way to replace char in source field with transforms.conf ? I saw CLEAN_KEYS but this attribute is only valid for search-time field extractions.

Props:
[mysource]
DATETIME_CONFIG = CURRENT
category = Custom
pulldown_type = 1
TRANSFORMS-EYI_Transform = e_source
CHARSET = AUTO

[e_source]
SOURCE_KEY = MetaData:Source
REGEX = ^source::(.*)$
FORMAT = filepath$1filepath$0
DEST_KEY = _raw

Event look like :
filepathD:\inetpub\LocalUser\MYFILE.TXTfilepath\xE1\xEC\xEB\x8C\x00\x00\x8C\x00\x0030.09.201601.01.0001x \x00NNYNNSAYX SAYX 2016-12-06-11.29.05.4154172016-12-06-13.09.42.541869\x00\x00\x00

Event should look like :
filepathD__inetpub_LocalUser_MYFILE.TXTfilepath\xE1\xEC\xEB\x8C\x00\x00\x8C\x00\x0030.09.201601.01.0001x \x00NNYNNSAYX SAYX 2016-12-06-11.29.05.4154172016-12-06-13.09.42.541869\x00\x00\x00

Re: Source Transform Replace '/' with '_'

lguinn2 — Wed, 28 Dec 2016 21:52:25 GMT

First - exactly what are you trying to do? Your transformation appears to attempt to manipulate both the source and the raw data.

If you are trying to change the actual source field for an event: there is no way to search-and-replace within the source field at indexing time.

If you are trying to change the characters in a file name that appears within the raw data of an event: you can do this. The rest of this answer explains how:

props.conf

[mysource]
DATETIME_CONFIG = CURRENT
category = Custom
pulldown_type = 1
CHARSET = AUTO
SEDCMD-abc = y/\/\:/__/

For more information about the SEDCMD, take a look at the Anonymize Data page in the documentation.

Re: Source Transform Replace '/' with '_'

ekremikizoglu — Tue, 29 Sep 2020 12:13:32 GMT

Hi,

thanks for your reply. I am sending these logs to 3rd application. So It does not know about data's file name. So I added source field to raw data to understand which file's data is.

I think your setting transforms all raw data . But i want to manupulate just part of raw data which is filename area.

Event look like :
filepathD:\inetpub\LocalUser\MYFILE.TXTfilepathrest of my raw data \0 bla bla:111

Event should look like :
filepathD__inetpub_LocalUser_MYFILE.TXTfilepathrest of my raw data \0 bla bla:111

Event should not look like :
filepathD__inetpub_LocalUser_MYFILE.TXTfilepathrest of my raw data _0 bla bla_111

Thank you.