Splunk Search

Event timestamp behavior is inconsistent when DATETIME_CONFIG = NONE

sylim_splunk
Splunk Employee
Splunk Employee

I want to use a file's modification timestamp as the Splunk timestamp for the events it contains.
Accordingly, I've set "DATETIME_CONFIG = NONE" in props.conf for the sourcetype. This props.conf is distributed to the forwarder and indexers..

The files are read with a "monitor" input on the forwarder.
Two different behaviors;
i) If Splunk on the forwarder is "not running" when the file is written, the file's events are timestamped as expected: i.e. the timestamp of the event matches the Modify time as reported by "stat" on the file.

ii) However, if Splunk is "running" when the file is written, the file's events are timestamped using "change time" of the input file.

Please advise how I can ensure that a file's events are always timestamped according to the file's modification time, regardless of whether the Splunk forwarder is running at the time the file is written.

Tags (1)
1 Solution

sylim_splunk
Splunk Employee
Splunk Employee

This appears to be caused by Kernel bugs as this has been reproduced constantly with Splunk version 7.2.4.2 and Linux, 3.10.0-957.12.1.el7.x86_64. Now with the latest version as of this writing, 3.10.0-1062.4.1.el7.x86_64 it works fine as described in the splunk doc.

It's been always reproduced with Ubuntu 16.04.1 but not with Ubuntu 16.04.5 - whoever sees this issue it'd be worth while to upgrade the OS kernel.

View solution in original post

sylim_splunk
Splunk Employee
Splunk Employee

This appears to be caused by Kernel bugs as this has been reproduced constantly with Splunk version 7.2.4.2 and Linux, 3.10.0-957.12.1.el7.x86_64. Now with the latest version as of this writing, 3.10.0-1062.4.1.el7.x86_64 it works fine as described in the splunk doc.

It's been always reproduced with Ubuntu 16.04.1 but not with Ubuntu 16.04.5 - whoever sees this issue it'd be worth while to upgrade the OS kernel.

Get Updates on the Splunk Community!

Introducing Edge Processor: Next Gen Data Transformation

We get it - not only can it take a lot of time, money and resources to get data into Splunk, but it also takes ...

Take the 2021 Splunk Career Survey for $50 in Amazon Cash

Help us learn about how Splunk has impacted your career by taking the 2021 Splunk Career Survey. Last year’s ...

Using Machine Learning for Hunting Security Threats

WATCH NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more for ...