Getting Data In

How to resolve latency between event time and index time?

uagraw01
Motivator

My index time is 7/6/20 3:37:42.210 PM 

My event time is 07/06/20 10:37:42.210 CDT

My TIME_FORMAT=%x %H:%M:%S.%3N%Z

But still, by referencing the above time, we can see the latency between index time and event time. Please suggest how to resolve this.

Labels (1)
Tags (1)
0 Karma

richgalloway
SplunkTrust
SplunkTrust
Is the index time in UTC or CDT? If the former, then there's no latency.
If the event time is in CDT then you have a latency of exactly 5 hours, which usually indicates a time zone setting error.
---
If this reply helps you, Karma would be appreciated.

uagraw01
Motivator

@richgalloway event time is CDT. So what i need to do.

0 Karma

richgalloway
SplunkTrust
SplunkTrust
Verify the source system has its time zone set correctly.
Verify the data source is sending the right timestamps.
Consider changing the TIME_FORMAT setting to drop the "%Z".
---
If this reply helps you, Karma would be appreciated.
0 Karma

uagraw01
Motivator

@richgalloway So you want me to use only TIME_FORMAT=%x %H:%M:%S.%3N and remove %Z?

0 Karma

richgalloway
SplunkTrust
SplunkTrust
If the timestamp in the event does not contain a time zone indication (or contains the wrong one) then "%Z" should not be part of TIME_FORMAT.
---
If this reply helps you, Karma would be appreciated.
0 Karma

uagraw01
Motivator

@richgalloway  Yes i will check, please let me know if source system and that source system logs are different timestamps, then it will create any latency ? And how to solve that?

0 Karma

richgalloway
SplunkTrust
SplunkTrust
Splunk indexes events as soon as they are received. An incorrect timestamp may generate a warning message, but will not cause index latency.
However, an incorrect TIME_FORMAT setting can cause Splunk to mis-interpret a timestamp and index it incorrectly, thus creating the appearance of latency.
---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Leveraging Detections from the Splunk Threat Research Team & Cisco Talos

 Stay ahead of today’s evolving threats with the combined power of the Splunk Threat Research Team (STRT) and ...

Splunk ITSI & Correlated Network Visibility

 Take Your Network Visibility to the Next LevelIn today’s complex IT environments, performance issues can stem ...

Splunk Classroom Chronicles: Training Tales and Testimonials (Episode 3)

Welcome back to Splunk Classroom Chronicles, our ongoing blog series that pulls back the curtain on Splunk ...