Monitoring Splunk

Date latency

LogUx
Motivator

I am receiving the logs from the forwarders and can see latency between index time and event time. We have difference between index time and event time is about 15 to 16 hours on more than 300 forwarders. How can  i fix this issue?

Labels (1)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

That's not (usually) a simple fix.  There are a variety of causes and finding the root cause will likely require intimate knowledge of your environment.

Some things to check include:

  • All servers are using NTP (or an equivalent time-sync service)
  • Time zones are set properly on each server
  • Event timestamps include a time zone indication or inputs.conf contains the TZ attribute
  • Props.conf has TIME_FORMAT attributes that correctly extract the time zone from event timestamps
  • All Splunk forwarders are always running
  • Any intermediate servers or processes are always running
  • Events are not cached by the generating server/process before they are sent to Splunk
---
If this reply helps you, Karma would be appreciated.

LogUx
Motivator

@richgalloway Is DATETIME_CONFIG = CURRENT will work ? 

0 Karma

richgalloway
SplunkTrust
SplunkTrust
It will "work" in that it will assign the current time to each event that arrives. It masks the latency problem. It makes old events look like new events and may throw off your reports.
---
If this reply helps you, Karma would be appreciated.
0 Karma

LogUx
Motivator

@richgalloway  Any other solution you can suggest to me. Because our thruput limit is set to 1024kb and that is fine . Any major issue we can fix this permanently.

 

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust
I offered 7 possible solutions in my first reply. Have you checked them?
---
If this reply helps you, Karma would be appreciated.
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Accelerate Splunk App Development with AI

  Learn how to automate the repetitive parts of your build cycle.   Watch On-Demand   Replay the Accelerate ...

Splunk Agent Observability: Measuring AI Value with Splunk Tokenomics on AWS

The AI landscape is moving faster than ever, and our latest 2026 State of Observability report provides the ...

Enterprise Security Watch Party: What’s New in 8.8 - Recap

Splunk Community HQ hosted a Splunk Enterprise Security Watch Party: What’s New In ES 8.8 last Thursday. I had ...