Deployment Architecture

Extract timestamp from CSV in UNIX time

msarro
Builder

Greetings! I'm still super new to splunk, so please be gentle :) I am trying to extract a timestamp from CSV records. A single CSV can contain N records. Each record has its own timestamp, but the timestamp is in UNIX time. Is there a way to index this so it can be correlated with other records that have their timestamps listed a different way?

For example (1270047982=march 31, 2010 15:06:22):

TimeStamp,Slot Number,Description,Card State,Redundancy State,Health Score
1270047982,slot 0,Signal Processing Unit,normal,initial,100
1270047982,slot 1,Signal Processing Unit,normal,active,100
Tags (2)

Lowell
Super Champion

I would recommend using an explicit setup for this kind of thing. Sure splunk "normally" get it right, but just letting splunk guess has burnt me a number of times (admittedly, on more complicated log events, but it could still happen). As simple as it is to setup a proper sourcetype, it just doesn't seem worth the risk in my mind. Also, Splunk can process you events faster if you tell it how to extract the timestamp.

You could add a props.conf entry something like this:

[my_source_type]
TIME_PREFIX = ^
TIME_FORMAT = %s
MAX_TIMESTAMP_LOOKAHEAD = 15
SHOULD_LINEMERGE = False

If you don't already have a sourcetype defined, and any CSV file really should have it's own sourcetype defined or you'll run into issues extracting fields; then you should take a look at the docs. I would start here: Override automatic source type assignment

Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...