Getting Data In

Extracting Timestamp From csv

RicoSuave
Builder

Hello. I'm having an issue when indexing a csv file. The format of the data is like this.

Employee,Date,Dept,Hours,Hour type,shift

Chuck Norris,Sat Jan 01 2011,10,hourtype,shift

etc

When i upload the file, splunk extracts each row as an event and does proper field extraction, but, it timestamps all of those events with the upload time of the file, intead of assigning it a timestamp using the date column. I tried the solution in this link to no effect: http://answers.splunk.com/questions/2672/defining-timestamp-on-data-from-csv-file

The odd thing is that another csv that gets indexed in nearly the same format, gets timestamped correctly. This csv is generated by a script. The only difference is that it has an additional column with a ticket number. Even after i tried modifying my csv with the same column, it still doesn't extract the timestamp correctly.

Tags (2)
0 Karma

hazekamp
Builder

You should be able to do this with a few index-time properties. TIME_PREFIX tells Splunk where to start looking for a time match, and TIME_FORMAT tells Splunk what format to use:

## props.conf
[<your_sourcetype>]
TIME_PREFIX = ([^,]+),
TIME_FORMAT = %a %b %d %Y

Additional references if needed:

http://www.splunk.com/base/Documentation/latest/Admin/Propsconf

http://docs.python.org/library/time.html

RicoSuave
Builder

So i have a csv with hundreds of lines in the same format i specified above, but the lines have different dates. When i index that csv, splunk doesn't extract the timestamp individually from the date column in the csv, instead it assigns all the events the same time. That time being when the file was indexed.

0 Karma

hazekamp
Builder

Joetron, I just indexed your sample log above with the TIME_PREFIX and TIME_FORMAT properties above and everything looks good. What specifically didn't work? Be sure to specify these properties on indexer or full forwarder.

0 Karma

RicoSuave
Builder

I tried this and it did not work. Is there anything else you recommend?

0 Karma

jbsplunk
Splunk Employee
Splunk Employee

Could you edit this post to include the configuration you are referring to in your props.conf? It would be helpful if we could see what you've attempted.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

We are excited to announce that the upcoming releases of Splunk Enterprise 10.2.x and Splunk Cloud Platform ...

Step into “Hunt the Insider: An Splunk ES Premier Mystery” to catch a cybercriminal ...

After a whole week of being on call, you fell asleep on your keyboard, and you hit a sequence of buttons that ...

SplunkTrust Application Period is Officially OPEN!

It's that time, folks! The application/nomination period for the 2026-2027 SplunkTrust is officially open. If ...