Can you help me with a timestamp extraction for monitoring log files?

[sarvesh_11]

I want to monitor a log file, a file in which there are a lot of time constraints. Date and time is defined within the log file.

Configuration in props.conf for default is set as DATETIME_CONFIG= \etc\datetime.xml.

Since i have to monitor the log file, just from 1 source, i am restricted to create any custom app or make any change in the default.

With the current set-up, what i am getting is, Splunk is reading the time from the content of the log file. While the requirement is to get the time at which the file is created or last modified. i.e to ignore the time that Splunk is reading from the events (log file).

I'm not sure, DATETIME_CONFIG = none will work if i define this in inputs.conf for that particular universal forwarder.

I am also not sure that this can be defined in Inputs.conf or not.

[woodcock · Accepted solution]

Do not modify anything inside of /opt/splunk/etc/system/.... You need not use a custom datetime.xml unless you single file has multiple timestamp variations (which is extremely uncommon/unlikely). Usually, you just need to use the following settings on the Indexers in a custom app's props.conf file:

[<YourSourcetypeHere>]
DATETIME_CONFIG = none

I think that this is a horrible idea and definitely THE WRONG THING TO DO. Why are you not using the timestamp in the event? There are many ways to fix those, if there is some kind of mistake in them. Fixing is vastly preferable to using this setting.

[sarvesh_11]

Hi @woodcock ,
can this be handled by outputs.conf?
if i define below in system/local/outputs.conf

[sourcetype / source]
DATETIME_CONFIG = none

[woodcock]

No. That setting must be in props.conf. Don't put it that directory, though.

[sarvesh_11]

Hi @woodcock ,
Thanks for replying.

1st) I was trying to make changes on /opt/SplunkUniversalForwarder/etc/system/local/, here i have created one props.conf, evaluating my sourcetype, with DATETIME_CONFIG=NONE

2nd) I cannot use the timestamp from the events because, the 1st issue what m facing is, i am not getting the data 1 file in single shot on search head, as it is taking the time stamp from the event. so it is showing forecasted time also, where as i just want to monitor this file every hour. i cannot increase my time range to past 24 hours. and also, the conclusion sentence , the keywords we wants to capture, appears at the last of every file, which is visible at 23:59:59, like if i am monitoring file at 9AM i wont be able to see that conclusion sentence in that file at that time.
SO what i am finally left is the index time of 1 file should be sync with the content of that file. i.r 1 source file should have only 1 timestamp. i.e the last modified file time.

[woodcock]

Nothing should ever be put into $SPLUNK_HOME/etc/system/local/. Create your own app and put it into $SPLUNK_HOME/etc/apps/<YourAppNameHere>/default/props.conf (that's where your inputs.conf and other files should go, too).

As far as using the wrong timestamp because there are multiple timestamps in each line, you should not be letting splunk guess; just tell it to use the correct one by using the settings to specify so.

[sarvesh_11]

Thanks Man @woodcock!
Appreciate your help.

Will do changes in props.conf itself for time stamp resolution.
[sourcetype]
DATETIME_CONFIG= none