Getting Data In

Data Input: Timestamps are read incorrectly for half of file.

chatham
Engager

I am currently trying to read in data from a .csv that has a timestamp column. When I upload the file and go to preview mode, I'm setting the custom timestamp field to my column. The splunk timestamp reads that data and sets the timestamp correctly for about half the values and then fails to read the data correctly for the other half

Example:

Timestamp               EVENT_DATETIME
9/6/14 12:00:00.000 PM  6.9.14 12:00
9/6/14 4:00:00.000 PM   6.9.14 16:00
9/6/14 8:00:00.000 PM   6.9.14 20:00
9/9/01 9:12:20.000 AM   7.9.14 0:00
9/9/01 12:02:00.000 PM  7.9.14 4:00

As you can see, the timestamp is formatted correctly for the first three entries then incorrectly for the last two.

No idea why this is happening...

Tags (2)
0 Karma

jrodman
Splunk Employee
Splunk Employee

The default timestamp behavior for Splunk is to guess the times and assume that they generally go forwards, because it was designed for logfile handling. For data that jumps back 13 years, you may have to set a TIME_FORMAT for the data, and you have have to adjust MAX_DAYS_AGO as well as the MAX_DIFF_SECS_AGO / MAX_DIFF_SECS_HENCE. DAYS_AGO puts a bound on the expected difference betwen file modtime and the timestamps, DIFF_SECS puts a bound on how far timestamps are expected to jump from event to event.

Start with a TIME_FORMAT though.

0 Karma
Get Updates on the Splunk Community!

CX Day is Coming!

Customer Experience (CX) Day is on October 7th!! We're so excited to bring back another day full of wonderful ...

Strengthen Your Future: A Look Back at Splunk 10 Innovations and .conf25 Highlights!

The Big One: Splunk 10 is Here!  The moment many of you have been waiting for has arrived! We are thrilled to ...

Now Offering the AI Assistant Usage Dashboard in Cloud Monitoring Console

Today, we’re excited to announce the release of a brand new AI assistant usage dashboard in Cloud Monitoring ...