topic Re: get timestamp from filename in Getting Data In

get timestamp from filename

olivier_romain — Mon, 28 Sep 2020 12:55:45 GMT

hello,

I am trying to retreive timestamp from filename. I have files named like

"disco_20120531.txt"

with content looking like:

"net0 family 'Web' application 'videosurf' path 'base.eth.8021q.ip.gre.ppp.ip.tcp.http.videosurf' rate 0 totbytes 25664 nb_packet 231 nb_uapp_cnx 25"

I try to set timestamp from filename "disco_20120531.txt" to 31/05/2012

However I couldn't make it. My app props.conf :

[source::/root/data/disco/daily/*]
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = false
pulldown_type = 1
TIME_PREFIX = disco_
TIME_FORMAT = %Y%m%d

This config works if the filename is added to the file content, but otherwise not. Time stamp is not found and splunk uses file mod time instead.

Does anyone has got an idea what's wrong?

Thanks in advance,

Olivier

Re: get timestamp from filename

lguinn2 — Sun, 09 Dec 2012 02:29:18 GMT

From the Splunk documentation here

"4. If no events in a source have a date, Splunk tries to find one in the source name or file name. (This requires that the events have a time, even though they don't have a date.)"

TIME_PREFIX and TIME_FORMAT are not used when parsing the date in a file name. They apply only when extracting the timestamp from an event.

Bottom line: Splunk will use your file modification date/time. I don't know any way around this, but perhaps someone else on this forum does. Or you could open a support ticket... The best option, if possible, is to add a full timestamp to every event.

Re: get timestamp from filename

olivier_romain — Mon, 28 Sep 2020 13:01:02 GMT

Thanks, I did set TIME_PREFIX and TIME_FORMAT so that splunk did not find any ts into the event itself. It does backup on the file update time, which is fine.

Re: get timestamp from filename

marcoscala — Thu, 15 May 2014 10:25:32 GMT

Hi Lisa,
I have the same problem too in Splunk 6.1, as many others, for a quite important prospect. I also had as last resort the idea of adding at the beginning of the _raw data the timestamp extracted from the source file, with date and time of the generation of the informations.

I only have a doubt: isn't timestamp assigned during the parsing phase before the Custom configurations in props.conf, like transforms and so on? We tried that but with no results...

Regards,
Marco