topic Re: transforms the source to remove timestamp in Splunk Search

transforms the source to remove timestamp

mataharry — Thu, 10 Mar 2011 06:52:00 GMT

I want to change the source filename for my data to remove the timestamp.

from mypath\to\my\folder\userentrypoint17_20110309T143708_170500.log to mypath\to\my\folder\userentrypoint17.log

the timestamp in the filename is not used, because the complete timestamp is precsent in each event.

Re: transforms the source to remove timestamp

yannK — Thu, 10 Mar 2011 06:55:46 GMT

Here is the method.

On the indexer side (or the regular forwarder)

in /local/props.conf
[sourcetypeofyourdata]
TRANSFORMS-changesource = removetimestamp

in /local/transforms.conf
[removetimetamp]
SOURCE_KEY = MetaData:Source
DEST_KEY = MetaData:Source
REGEX = (.*?)(_\d{8}T\d{6}_\d{6})(\.log)
#use a regex to extract the filename
FORMAT = source::$1$3

to explain here is the regex in action : mypath\userentrypoint17_20110309T143708_170500.log
is cut in 
$1: mypath\userentrypoint17
$2: _20110309T143708_170500
$3: .log 
and we throw away the $2

Re: transforms the source to remove timestamp

Lowell — Tue, 02 Aug 2011 17:38:46 GMT

You may find some of the transformer examples here helpful as well:

http://splunk-base.splunk.com/answers/3470/consolidate-similarly-named-log-files-into-a-single-source/4420