topic Re: Why are some events being indexed with the wrong timestamp? in Getting Data In

Why are some events being indexed with the wrong timestamp?

alexanderadler — Tue, 08 Jan 2019 08:24:15 GMT

Hi! I have a big Splunk enterprise environment, but I'm experiencing a strange issue where some events are losing part of their timestamp. These are the timestamps for the events

2019-01-08 07:05:32,776 StatisticMessage ,
2019-01-08 07:05:33,166 StatisticMessage , 
2019-01-08 07:05:33,401 StatisticMessage ,

and this is the props.conf file that is deployed.

[NameOfSourceType]
MAX_TIMESTAMP_LOOKAHEAD = 60
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\n\r]+)\d{4}\-\d{2}\-\d{2}\s\d{2}\:\d{2}\:\d{2}\,\d{3}
TRUNCATE = 1999999

But some events are getting part of their time scrambled as shown below.

Re: Why are some events being indexed with the wrong timestamp?

lakshman239 — Tue, 29 Sep 2020 22:36:28 GMT

I assume you want to extract the date and time from the first 24 chars, so try the below:
//limiting the lookahead, adding prefix and changing line breaker to be positive lookahead

[NameOfSourceType]
MAX_TIMESTAMP_LOOKAHEAD = 24
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\n\r]+)(?=\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\,\d{3}\s)
TRUNCATE = 999999

Re: Why are some events being indexed with the wrong timestamp?

richgalloway — Tue, 08 Jan 2019 13:42:19 GMT

Have you seen any DateParserVerbose warnings in splunkd.log about this source/sourcetype?

Re: Why are some events being indexed with the wrong timestamp?

alexanderadler — Fri, 11 Jan 2019 09:21:36 GMT

Hi! Tried this but still the same outcome.

Re: Why are some events being indexed with the wrong timestamp?

lakshman239 — Fri, 11 Jan 2019 14:03:39 GMT

send sample file with a few events [ pls remove/mask sensitive data]