topic Re: Time stamp, incorrect date for later events in Getting Data In

Time stamp, incorrect date for later events

atifshaukat — Tue, 17 Jul 2012 11:29:48 GMT

I am continuously indexing data from CSV file. Events only have time stamp without date. Splunk has automatically extracted time stamp and used file modified date to use it as date with in time stamp. Please find below example.
FIELDS = "File", "Time", "Copy", "Open", "Save", "Close"

3 » 16/07/2012 23:58:25.000 50K.xls 23:58:25 0.0156 0.7813 0.6719 0.0469
4 » 16/07/2012 23:58:19.000 50K.ppt 23:58:19 0.0156 0.9219 0.5625 0.0313

So if the next even arrives at 00:01:02, the splunk will assign it a date of 16/07/2012 instead of automatically detecting a day change and assign it date of 17/07/2012. Please can you help me to identify that how this problem can fixed.

Please find below an examples where following events should have assigned date of 17/07/2012 but they were assigned incorrect date of 16/07/2012

16/07/2012 00:09:02.000 500K.xls 00:09:02 0.0150 4.0161 1.4529 0.3600
3902 » 16/07/2012 00:08:52.000 500k.ppt 00:08:52 0.0160 5.9070 4.6250 0.0309

Thanks

Re: Time stamp, incorrect date for later events

rturk — Mon, 28 Sep 2020 12:06:18 GMT

Hi atifshaukat,

Just to clarify, your CSV looks something like this (spaced for readability), with no date entered into the events:

File,      Time,       Copy,   Open,   Save,   Close
50K.xls,   23:58:25,   0.0156, 0.7813, 0.6719, 0.0469 
50K.ppt,   23:58:19,   0.0156, 0.9219, 0.5625, 0.0313 
500K.xls,  00:09:02,   0.0150, 4.0161, 1.4529, 0.3600 
500k.ppt,  00:08:52,   0.0160, 5.9070, 4.6250, 0.0309

Splunk will always trust the log file as a "source of truth" for the timestamp of an event. In the event that there is a time but no date in the event it will attempt to derive the date from the filename of the source or filename, so what is the filename of the CSV you are indexing?

More information on how Splunk assigns timestamps may be found here: http://docs.splunk.com/Documentation/Splunk/latest/Data/HowSplunkextractstimestamps

As a side note, I'd be looking at whatever is generating the log (whether that's a script, application, etc) to see whether the logging can be modified to include the date, or whether you can configure some kind of rollover at midnight where new events are written into a new CSV file (eg. file_operations_20120617.csv).

Hope this has been of some help 🙂

Re: Time stamp, incorrect date for later events

atifshaukat — Tue, 17 Jul 2012 12:55:59 GMT

Your assumptions are correct about csv. File name includes that so it is definitely extracting it from there. Is possible to use File modified time as a source?

Re: Time stamp, incorrect date for later events

rturk — Tue, 17 Jul 2012 13:00:09 GMT

From the posted link:

For file sources, if no time or date can be identified in the file name, Splunk uses the file's modification time.

So if you can remove the date string from the the CSV filename, fix the file naming/rotation, or (preferred) get the date entered into the log file, then you should be good to go 🙂

Re: Time stamp, incorrect date for later events

nitesh218ss — Tue, 12 May 2015 06:16:38 GMT

Hi in my log event and filename date is not present i want give a fix date to log so what is do ?