Getting Data In

How to resolve latency between event time and index time?

LogUx
Motivator

My index time is 7/6/20 3:37:42.210 PM 

My event time is 07/06/20 10:37:42.210 CDT

My TIME_FORMAT=%x %H:%M:%S.%3N%Z

But still, by referencing the above time, we can see the latency between index time and event time. Please suggest how to resolve this.

Labels (1)
Tags (1)
0 Karma

richgalloway
SplunkTrust
SplunkTrust
Is the index time in UTC or CDT? If the former, then there's no latency.
If the event time is in CDT then you have a latency of exactly 5 hours, which usually indicates a time zone setting error.
---
If this reply helps you, Karma would be appreciated.

LogUx
Motivator

@richgalloway event time is CDT. So what i need to do.

0 Karma

richgalloway
SplunkTrust
SplunkTrust
Verify the source system has its time zone set correctly.
Verify the data source is sending the right timestamps.
Consider changing the TIME_FORMAT setting to drop the "%Z".
---
If this reply helps you, Karma would be appreciated.
0 Karma

LogUx
Motivator

@richgalloway So you want me to use only TIME_FORMAT=%x %H:%M:%S.%3N and remove %Z?

0 Karma

richgalloway
SplunkTrust
SplunkTrust
If the timestamp in the event does not contain a time zone indication (or contains the wrong one) then "%Z" should not be part of TIME_FORMAT.
---
If this reply helps you, Karma would be appreciated.
0 Karma

LogUx
Motivator

@richgalloway  Yes i will check, please let me know if source system and that source system logs are different timestamps, then it will create any latency ? And how to solve that?

0 Karma

richgalloway
SplunkTrust
SplunkTrust
Splunk indexes events as soon as they are received. An incorrect timestamp may generate a warning message, but will not cause index latency.
However, an incorrect TIME_FORMAT setting can cause Splunk to mis-interpret a timestamp and index it incorrectly, thus creating the appearance of latency.
---
If this reply helps you, Karma would be appreciated.
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Your Feedback. Our Roadmap. Visit the PX Feedback Booth at .conf26

You use Splunk every day, come and help shape what's next.  Save Your Seat: Product-Focused Sessions at ...

Agentic SOC Triage: Investigating Splunk ES Notables with MCP Server and a Local LLM

The Problem: Too Many Alerts, Too Little Context Security operations teams running Splunk Enterprise Security ...

Painting a Clearer Picture: Creating Cross-Domain Visibility with AI Canvas

Watch Now Painting a Clearer Picture: Creating Cross-Domain Visibility with AI Canvas     Do you ever feel ...