Splunk Search

Event timestamp behavior is inconsistent when DATETIME_CONFIG = NONE

sylim_splunk
Splunk Employee
Splunk Employee

I want to use a file's modification timestamp as the Splunk timestamp for the events it contains.
Accordingly, I've set "DATETIME_CONFIG = NONE" in props.conf for the sourcetype. This props.conf is distributed to the forwarder and indexers..

The files are read with a "monitor" input on the forwarder.
Two different behaviors;
i) If Splunk on the forwarder is "not running" when the file is written, the file's events are timestamped as expected: i.e. the timestamp of the event matches the Modify time as reported by "stat" on the file.

ii) However, if Splunk is "running" when the file is written, the file's events are timestamped using "change time" of the input file.

Please advise how I can ensure that a file's events are always timestamped according to the file's modification time, regardless of whether the Splunk forwarder is running at the time the file is written.

Tags (1)
1 Solution

sylim_splunk
Splunk Employee
Splunk Employee

This appears to be caused by Kernel bugs as this has been reproduced constantly with Splunk version 7.2.4.2 and Linux, 3.10.0-957.12.1.el7.x86_64. Now with the latest version as of this writing, 3.10.0-1062.4.1.el7.x86_64 it works fine as described in the splunk doc.

It's been always reproduced with Ubuntu 16.04.1 but not with Ubuntu 16.04.5 - whoever sees this issue it'd be worth while to upgrade the OS kernel.

View solution in original post

sylim_splunk
Splunk Employee
Splunk Employee

This appears to be caused by Kernel bugs as this has been reproduced constantly with Splunk version 7.2.4.2 and Linux, 3.10.0-957.12.1.el7.x86_64. Now with the latest version as of this writing, 3.10.0-1062.4.1.el7.x86_64 it works fine as described in the splunk doc.

It's been always reproduced with Ubuntu 16.04.1 but not with Ubuntu 16.04.5 - whoever sees this issue it'd be worth while to upgrade the OS kernel.

Get Updates on the Splunk Community!

Splunk Observability for AI

Don’t miss out on an exciting Tech Talk on Splunk Observability for AI!Discover how Splunk’s agentic AI ...

Splunk Enterprise Security 8.x: The Essential Upgrade for Threat Detection, ...

Watch On Demand the Tech Talk on November 6 at 11AM PT, and empower your SOC to reach new heights! Duration: ...

Splunk Observability as Code: From Zero to Dashboard

For the details on what Self-Service Observability and Observability as Code is, we have some awesome content ...