I want to change the source filename for my data to remove the timestamp.
from mypath\to\my\folder\userentrypoint17_20110309T143708_170500.log to mypath\to\my\folder\userentrypoint17.log
the timestamp in the filename is not used, because the complete timestamp is precsent in each event.
Here is the method.
On the indexer side (or the regular forwarder)
in /local/props.conf
[sourcetypeofyourdata]
TRANSFORMS-changesource = removetimestamp
in /local/transforms.conf
[removetimetamp]
SOURCE_KEY = MetaData:Source
DEST_KEY = MetaData:Source
REGEX = (.*?)(_\d{8}T\d{6}_\d{6})(\.log)
#use a regex to extract the filename
FORMAT = source::$1$3
to explain here is the regex in action : mypath\userentrypoint17_20110309T143708_170500.log
is cut in
$1: mypath\userentrypoint17
$2: _20110309T143708_170500
$3: .log
and we throw away the $2
You may find some of the transformer examples here helpful as well:
Here is the method.
On the indexer side (or the regular forwarder)
in /local/props.conf
[sourcetypeofyourdata]
TRANSFORMS-changesource = removetimestamp
in /local/transforms.conf
[removetimetamp]
SOURCE_KEY = MetaData:Source
DEST_KEY = MetaData:Source
REGEX = (.*?)(_\d{8}T\d{6}_\d{6})(\.log)
#use a regex to extract the filename
FORMAT = source::$1$3
to explain here is the regex in action : mypath\userentrypoint17_20110309T143708_170500.log
is cut in
$1: mypath\userentrypoint17
$2: _20110309T143708_170500
$3: .log
and we throw away the $2