splunk event time and timestamp on log file is not matching.

[sim_tcr]

splunk event time and timestamp on log file is not matching.
Our log file has below entry for timestamp
"2020-02-20 10:14:59.363"
But that time and splunk time not matching.
How can I fix it?

Below is the the sourcetype is set,
[name]

SHOULD_LINEMERGE = True

BREAK_ONLY_BEFORE = Id-

SHOULD_LINEMERGE=false

[nickhills]

That timestamp is a long way into the log. Splunks default is to only look at the first 128 bytes to find the timestamp. In your example its much further into the message.

It's difficult to give too much guidance because your data is heavily redacted, but see if you can use some regex to set an anchor on something consistent in your logs and use that to set a TIME_PREFIX

TIME_PREFIX = <regular expression>
* If set, Splunk software scans the event text for a match for this regex
  in event text before attempting to extract a timestamp.
* The timestamping algorithm only looks for a timestamp in the text
  following the end of the first regex match.
* For example, if 'TIME_PREFIX' is set to "abc123", only text following the
  first occurrence of the text abc123 is used for timestamp extraction.
* If the 'TIME_PREFIX' cannot be found in the event text, timestamp extraction
  does not occur.
* Default: empty string

Then set the MAX_TIMESTAMP_LOOKAHEAD to run from the prefix:

MAX_TIMESTAMP_LOOKAHEAD = <integer>
* The number of characters into an event Splunk software should look
  for a timestamp.
* This constraint to timestamp extraction is applied from the point of the
  'TIME_PREFIX'-set location.
* For example, if 'TIME_PREFIX' positions a location 11 characters into the
  event, and MAX_TIMESTAMP_LOOKAHEAD is set to 10, timestamp extraction is
  constrained to characters 11 through 20.
* If set to 0 or -1, the length constraint for timestamp recognition is
  effectively disabled. This can have negative performance implications
  which scale with the length of input lines (or with event size when
  'LINE_BREAKER' is redefined for event splitting).
* Default: 128

I would even explicitly configure the TIME_FORMAT format too:

TIME_FORMAT = <strptime-style format>
* Specifies a "strptime" format string to extract the date.
* "strptime" is an industry standard for designating time formats.
* For more information on strptime, see "Configure timestamp recognition" in
  the online documentation.
* TIME_FORMAT starts reading after the TIME_PREFIX. If both are specified,
  the TIME_PREFIX regex must match up to and including the character before
  the TIME_FORMAT date.
* For good results, the <strptime-style format> should describe the day of
  the year and the time of day.
* Default: empty string

A "BEST GUESS" example (and probably less than ideal) would be to look for "ID" in your message and use that as a prefix (assuming ID appears in each log)
in props.conf

[name]
TIME_PREFIX =ID
MAX_TIMESTAMP_LOOKAHEAD=60
TIME_FORMAT=%Y-%m-%d %H:%M:%S.%N3

The above tells Splunk to "start looking for the timestamp within the next 60 bytes after it sees 'ID' in the logs."

[gcusello]

Hi @sim_tcr,
what's the problem: does Splunk take a wrong timestamp from another time of the events or read in wrong way the timestamp?
In the first case, you should set, for your sourcetype, the TIME_PREFIX option, to be sure that Splunk uses the correct timestamp and not another one in the event.
If you could share an example of your logs I could help you.

Ciao.
Giuseppe

[sim_tcr]

here is sample event from log file.
Issue is, in below event there is a timestamp. That and splunk time not matching.

"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","xx.xxx.xxx.xx","XXXXXXXXXX","200","","","XXXX","xxxxxxxxxxx-xxx.xx.com","","xxxx xxxx xxxxx","xxxxx","http://xxxx.xxx.com/xxxxxxxxxxx/xxxxx/service/bac/v1","xxxxxxxxxxx","xxxxxx","xxxxxxxx","XXXXXXX","x... 10:14:59.363"***,"2","0","88","97","0","0","6","0","1","0","0","0","0","SOAP","false","40000","false"

[gcusello]

Hi @sim_tcr,
if the event is always the same you could use:

TIME_PREFIX = ^(\"[^\"]*\",){25}\"

or if you're not sure about your event, you could use:

TIME_PREFIX = \"

and

TIME_FORMAT = %Y-%m-%d %H:%M:%S\.%3N

Ciao.
Giuseppe