topic Re: Incorrect timestamp when ingesting csv data in Getting Data In

Incorrect timestamp when ingesting csv data

jg91 — Fri, 18 Oct 2024 07:14:11 GMT

Hi,

I’m trying to ingest CSV data (without a timestamp) using a Universal Forwarder (UF) running in a fresh container.

When I attempt to ingest the data, I encounter the following warning in the _internal index, and the data ends up being ingested with a timestamp from 2021. This container has not previously ingested any data, so I’m unsure why it defaults to this date.

10-18-2024 03:42:00.942 +0000 WARN  DateParserVerbose [1571 structuredparsing] - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Wed Jan 13 21:06:54 2021). Context: source=/var/data/sample.csv|host=splunk-uf|csv|6215

Can someone explain why this date is being applied, and how I can prevent this issue?

Re: Incorrect timestamp when ingesting csv data

gcusello — Fri, 18 Oct 2024 07:19:25 GMT

Hi @jg91 ,

if your csv doesn't contain any timestamp, Splunk can assign the timestamp of the index time or the timestamp from the previous event.

probably it's assigned the second one.

I hint to specify in props.conf that the timestamp is the current time:

DATETIME_CONFIG = CURRENT

as described at https://docs.splunk.com/Documentation/Splunk/9.3.1/Admin/Propsconf#Timestamp_extraction_configuration

Ciao.

Giuseppe

Re: Incorrect timestamp when ingesting csv data

jg91 — Fri, 18 Oct 2024 07:48:02 GMT

Thank you, but my question is why it defaults to a timestamp from 2021, especially since this is a freshly created container/pod with no prior data ingested. Why is it using that specific date?

Re: Incorrect timestamp when ingesting csv data

gcusello — Fri, 18 Oct 2024 07:56:39 GMT

Hi @jg91 ,

I don't know your data, maybe there's some numeric field that can be interpretated as a timestamp, or there's a previous event of 2021, I don't know.

But using the above configuration you should solve.

Ciao.

Giuseppe

Re: Incorrect timestamp when ingesting csv data

PickleRick — Fri, 18 Oct 2024 20:34:16 GMT

Hard to say without knowing your exact data and config. But Splunk does tend to try to guess the time format sometimes and it's usually not the best idea to let it. So if you don't have timestamps in your data it's best to explicitly configure your sourcetype so that Splunk doesn't guess but blindly assumes it's the current timestamp (as @gcusello already showed)