topic Re: splunk _time not matching with timestamp in Splunk Search

splunk _time not matching with timestamp

nawazns5038 — Tue, 29 Sep 2020 17:46:32 GMT

Hi,

the log has timestamp like this "time":"2018-01-22 13:43:40.0"

props.conf :
TIME_FORMAT = %F %T.%3N
TIME_PREFIX = "time":\"
MAX_TIMESTAMP_LOOKAHEAD = 25

that was the props conf used. The setting worked fine while testing. After indexing the data the timestamp shown in Splunk or _time has come upto "11/28/17 4:06:53.568 PM" . which is even no where present in the event.

How can this be resolved. Please help.

Re: splunk _time not matching with timestamp

micahkemp — Tue, 23 Jan 2018 04:08:51 GMT

Can you paste the full _raw of the event in question? Sometimes there are multiple instances of timestamps on one message, which can confuse Splunk.

Re: splunk _time not matching with timestamp

p_gurav — Tue, 23 Jan 2018 04:18:56 GMT

Hi nawazns5038,
Could you please provide sample events? Also in TIME_PREFIX try changing rex to \"time\"\:\"

Re: splunk _time not matching with timestamp

nawazns5038 — Tue, 23 Jan 2018 20:21:38 GMT

It's really strange ... "time":"2018-01-21 05:42:34.0"
There is only time field and all the events are of the same format...Some are taking the correct value and some are not even in the range or just picking up some random timestamp which is not even present in the event
The data is being sent from S3 input from AWS addon

Re: splunk _time not matching with timestamp

nawazns5038 — Tue, 23 Jan 2018 20:36:26 GMT

@p_gaurav

There is only time field and all the events are of the same format...Some are taking the correct value and some are not even in the range or just picking up some random timestamp which is not even present in the event
The data is being sent from S3 input from AWS addon

Re: splunk _time not matching with timestamp

nickhills — Tue, 23 Jan 2018 20:52:07 GMT

Are the events 'breaking' properly? Ie, one valid json block per event?
Also is it aws generated data (s3/cloudwatch logs) etc, or your own log data?

Re: splunk _time not matching with timestamp

nawazns5038 — Tue, 23 Jan 2018 23:22:26 GMT

Ya breaking is proper and props has been tested with the sample data as well. It is only one Json block.
Data is pulled from S3 buckets and it is not AWS default data . and it is pulling lots of .gz files which has json files in it
Is it a problem related to the Addon ?

Re: splunk _time not matching with timestamp

p_gurav — Wed, 24 Jan 2018 07:07:14 GMT

Can you check _internal logs for particular S3 input? Is there any timestamp related warning or error?