Getting Data In

Why are some events being indexed with the wrong timestamp?

alexanderadler
New Member

Hi! I have a big Splunk enterprise environment, but I'm experiencing a strange issue where some events are losing part of their timestamp. These are the timestamps for the events

2019-01-08 07:05:32,776 StatisticMessage ,
2019-01-08 07:05:33,166 StatisticMessage , 
2019-01-08 07:05:33,401 StatisticMessage , 

alt text
and this is the props.conf file that is deployed.

[NameOfSourceType]
MAX_TIMESTAMP_LOOKAHEAD = 60
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\n\r]+)\d{4}\-\d{2}\-\d{2}\s\d{2}\:\d{2}\:\d{2}\,\d{3}
TRUNCATE = 1999999

But some events are getting part of their time scrambled as shown below.

alt text

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Have you seen any DateParserVerbose warnings in splunkd.log about this source/sourcetype?

---
If this reply helps you, Karma would be appreciated.
0 Karma

lakshman239
Influencer

I assume you want to extract the date and time from the first 24 chars, so try the below:
//limiting the lookahead, adding prefix and changing line breaker to be positive lookahead

[NameOfSourceType]
MAX_TIMESTAMP_LOOKAHEAD = 24
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\n\r]+)(?=\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\,\d{3}\s)
TRUNCATE = 999999

0 Karma

alexanderadler
New Member

Hi! Tried this but still the same outcome.

0 Karma

lakshman239
Influencer

send sample file with a few events [ pls remove/mask sensitive data]

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...