Archive
Highlighted

Why does Splunk recognize the timestamp only for specific dates?

Engager

Hello,

I have a csv file with data from 2010 until 2017.

Splunk seems to parse the timestamp correctly for most of the data but when the date is from 2010 or 2011 or 2012, I see the message: Could not use timestamp to parse the data from "".
i.e. Could not use timestamp to parse the data from "1/21/2010".

The format of date in the csv file is month/day/year.

Why does Splunk recognize the timestamp when the date is 1/20/2017 23:00:00 PM but it doesn't recognizes the timestamp when the date is 1/21/2010 11:00:00 AM?

Sample of data:
Date Type Latitude Longitude Id
1/21/2010 11:00 Dry Cargo 39.3869634 22.9385489 29
1/22/2010 8:00 Dry Cargo 39.3675609 22.9491659 30
1/23/2010 13:30 Dry Cargo 39.367539 22.9229295 31
1/24/2010 9:00 Refrigerated Cargo 39.3686508 22.9414365 32
1/26/2010 18:00 Dry Cargo 39.3766097 22.9603403 33
1/26/2010 17:00 Dry Cargo 39.3557886 22.9581058 34
1/27/2010 10:00 Refrigerated Cargo 39.3799523 22.9232278 35
1/27/2010 12:00 Dry Cargo 39.3647131 22.9517557 36

Thank you in advance!

Tags (1)
0 Karma
Highlighted

Re: Why does Splunk recognize the timestamp only for specific dates?

Influencer

Are you ingesting that file somehow or just inputing it as a lookup?

0 Karma
Highlighted

Re: Why does Splunk recognize the timestamp only for specific dates?

Engager

I uploaded the csv file from my computer.

0 Karma
Highlighted

Re: Why does Splunk recognize the timestamp only for specific dates?

SplunkTrust
SplunkTrust

You need to set base configs which tell Splunk how to read the timestamp

Add this to your props.conf and restart the splunkd service

[YOUR_SOURCETYPE]
TIME_PREFIX=^
TIME_FORMAT=%m/%e/%Y %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD=18
0 Karma
Highlighted

Re: Why does Splunk recognize the timestamp only for specific dates?

SplunkTrust
SplunkTrust

What is your MAXDAYSAGO setting? I would expect a different error message if this was the cause, but it's worth changing it to 5000 or so to see if it helps. The default setting is 2000, which means Splunk will reject timestamps more than 5 years old.

---
If this reply helps you, an upvote would be appreciated.
0 Karma
Highlighted

Re: Why does Splunk recognize the timestamp only for specific dates?

Engager

Thanks for the advise.
I have set the MAXDAYSAGO to 5000 in props.conf.
Actually, my props.conf is:
[data]
DATETIMECONFIG =
MAX
DAYSAGO = 5000
INDEXED
EXTRACTIONS = csv
KVMODE = none
NO
BINARYCHECK = true
SHOULD
LINEMERGE = false
TIMESTAMPFIELDS = Date
category = Structured
description = Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled = false
pulldown
type = true

However, it doesn't work. Splunk still doesn't recognize dates from 2010, 2011 and 2012. 😞

0 Karma
Highlighted

Re: Why does Splunk recognize the timestamp only for specific dates?

SplunkTrust
SplunkTrust

It's because you don't have TIME_PREFIX or TIME_FORMAT set.. I gave you the correct stanza in my answer above..

0 Karma
Highlighted

Re: Why does Splunk recognize the timestamp only for specific dates?

Engager

Hello skoelpin,

Thank you for the answer.

I have tried what you suggested, but Splunk cannot read the timestamp.
I still see the message: Could not use timestamp to parse the data from "".

Is there anything else that I can try?

0 Karma
Highlighted

Re: Why does Splunk recognize the timestamp only for specific dates?

SplunkTrust
SplunkTrust

Did you restart Splunk after modifying the config file?

---
If this reply helps you, an upvote would be appreciated.
0 Karma
Highlighted

Re: Why does Splunk recognize the timestamp only for specific dates?

Engager

Yes, I restarted Splunk.
Every time I do a change in the conf files, I restart Splunk.

0 Karma