Getting Data In
Highlighted

Why are some events being indexed with the wrong timestamp?

New Member

Hi! I have a big Splunk enterprise environment, but I'm experiencing a strange issue where some events are losing part of their timestamp. These are the timestamps for the events

2019-01-08 07:05:32,776 StatisticMessage ,
2019-01-08 07:05:33,166 StatisticMessage , 
2019-01-08 07:05:33,401 StatisticMessage , 

alt text
and this is the props.conf file that is deployed.

[NameOfSourceType]
MAX_TIMESTAMP_LOOKAHEAD = 60
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\n\r]+)\d{4}\-\d{2}\-\d{2}\s\d{2}\:\d{2}\:\d{2}\,\d{3}
TRUNCATE = 1999999

But some events are getting part of their time scrambled as shown below.

alt text

0 Karma
Highlighted

Re: Why are some events being indexed with the wrong timestamp?

SplunkTrust
SplunkTrust

I assume you want to extract the date and time from the first 24 chars, so try the below:
//limiting the lookahead, adding prefix and changing line breaker to be positive lookahead

[NameOfSourceType]
MAXTIMESTAMPLOOKAHEAD = 24
TIMEPREFIX = ^
TIME
FORMAT = %Y-%m-%d %H:%M:%S,%3N
SHOULDLINEMERGE = false
LINE
BREAKER = ([\n\r]+)(?=\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\,\d{3}\s)
TRUNCATE = 999999

0 Karma
Highlighted

Re: Why are some events being indexed with the wrong timestamp?

New Member

Hi! Tried this but still the same outcome.

0 Karma
Highlighted

Re: Why are some events being indexed with the wrong timestamp?

SplunkTrust
SplunkTrust

send sample file with a few events [ pls remove/mask sensitive data]

0 Karma
Highlighted

Re: Why are some events being indexed with the wrong timestamp?

SplunkTrust
SplunkTrust

Have you seen any DateParserVerbose warnings in splunkd.log about this source/sourcetype?

---
If this reply helps you, an upvote would be appreciated.
0 Karma