Hello
in my organisation we have few kinds of log format
one of them does not have the year in the time stamp so the event looks like:
Jun 6 02:32:43 : Info:Environment.cpp:27: MARINERVAR
this is causes me lots of problems in the report since splunk does not now what to do with this timestamp and i have cases where i get future time 😕
at the begging of the file i have full date
it looks like :
Thu Jun 6 02:32:43 CDT 2019
is it possible to use the year from the begging of the file and add it to timestamp at index time ?
thanks
A missing year to your timestamp should not cause any problems if you have set up timestamp recognition in your props.conf correctly.
Try using the following parameters in props.conf for your relevant sourcetype (assuming the timestamp is at the beginning of your event):
[yoursourcetype]
TIME_PREFIX = ^
TIME_FORMAT = %b %d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 16
This should tell Splunk how to read your timestamp correctly and not produce any future-timestamped events, as it will try to stay as close to the current time as possible.
this is the configuration i have :
[fdm_f123_systemLog]
BREAK_ONLY_BEFORE = ^\w\s\d+\s\d{2}:\d{2}:\d{2}
DATETIME_CONFIG =
LINE_BREAKER = ([\r\n]+)
MAX_TIMESTAMP_LOOKAHEAD = 15
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIME_FORMAT = %b %e %H:%M:%S
TIME_PREFIX = ^
TRUNCATE = 0
category = Custom
disabled = false
pulldown_type = 1
is it ok ?
the problem is not only the future date
the problem is that it is possible that i will have events from 2018 at the same file
is it possible to take the year from somewhere else ?
I am not 100% sure about this, but you can try to use an additional datetime.xml
to extract the year from the filename. I am not aware of any method to exract the time (which is an index-time operation, hence done per-event) from any event earlier in the file.
Check https://docs.splunk.com/Documentation/Splunk/7.3.0/Data/Configuredatetimexml for details of the datetime.xml usage.
taking it from file name will not help in that case since i can have events from year before
In that case the only possibility would be - as bad as it sounds - to check your logging ...
If you get logs in one file that are years apart, I would personally consider the logging itself to be crap.
yeah i know.. it is not on my side
thanks