Getting Data In

Incorrect timestamp when ingesting csv data

jg91
Path Finder

Hi,

I’m trying to ingest CSV data (without a timestamp) using a Universal Forwarder (UF) running in a fresh container.

When I attempt to ingest the data, I encounter the following warning in the _internal index, and the data ends up being ingested with a timestamp from 2021. This container has not previously ingested any data, so I’m unsure why it defaults to this date.

10-18-2024 03:42:00.942 +0000 WARN  DateParserVerbose [1571 structuredparsing] - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Wed Jan 13 21:06:54 2021). Context: source=/var/data/sample.csv|host=splunk-uf|csv|6215

 

Can someone explain why this date is being applied, and how I can prevent this issue?

 

Labels (3)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @jg91 ,

if your csv doesn't contain any timestamp, Splunk can assign the timestamp of the index time or the timestamp from the previous event.

probably it's assigned the second one.

I hint to specify in props.conf that the timestamp is the current time:

DATETIME_CONFIG = CURRENT

as described at https://docs.splunk.com/Documentation/Splunk/9.3.1/Admin/Propsconf#Timestamp_extraction_configuratio...

Ciao.

Giuseppe

0 Karma

jg91
Path Finder

Thank you, but my question is why it defaults to a timestamp from 2021, especially since this is a freshly created container/pod with no prior data ingested. Why is it using that specific date?

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Hard to say without knowing your exact data and config. But Splunk does tend to try to guess  the time format sometimes and it's usually not the best idea to let it. So if you don't have timestamps in your data it's best to explicitly configure your sourcetype so that Splunk doesn't guess but blindly assumes it's the current timestamp (as @gcusello already showed)

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @jg91 ,

I don't know your data, maybe there's some numeric field that can be interpretated as a timestamp, or there's a previous event of 2021, I don't know.

But using the above configuration you should solve.

Ciao.

Giuseppe

Get Updates on the Splunk Community!

Cultivate Your Career Growth with Fresh Splunk Training

Growth doesn’t just happen—it’s nurtured. Like tending a garden, developing your Splunk skills takes the right ...

Introducing a Smarter Way to Discover Apps on Splunkbase

We’re excited to announce the launch of a foundational enhancement to Splunkbase: App Tiering.  Because we’ve ...

How to Send Splunk Observability Alerts to Webex teams in Minutes

As a Developer Evangelist at Splunk, my team and I are constantly tinkering with technology to explore its ...