Date latency

[uagraw01]

I am receiving the logs from the forwarders and can see latency between index time and event time. We have difference between index time and event time is about 15 to 16 hours on more than 300 forwarders. How can  i fix this issue?

[richgalloway]

That's not (usually) a simple fix.  There are a variety of causes and finding the root cause will likely require intimate knowledge of your environment.

Some things to check include:

• All servers are using NTP (or an equivalent time-sync service)
• Time zones are set properly on each server
• Event timestamps include a time zone indication or inputs.conf contains the TZ attribute
• Props.conf has TIME_FORMAT attributes that correctly extract the time zone from event timestamps
• All Splunk forwarders are always running
• Any intermediate servers or processes are always running
• Events are not cached by the generating server/process before they are sent to Splunk

[uagraw01]

@richgalloway Is DATETIME_CONFIG = CURRENT will work ? 

[richgalloway]

It will "work" in that it will assign the current time to each event that arrives. It masks the latency problem. It makes old events look like new events and may throw off your reports.

[uagraw01]

@richgalloway  Any other solution you can suggest to me. Because our thruput limit is set to 1024kb and that is fine . Any major issue we can fix this permanently.

[richgalloway]

I offered 7 possible solutions in my first reply. Have you checked them?