topic Re: DATETIME_CONFIG=NONE DateParserVerbose - Failed to parse timestamp in Getting Data In

DATETIME_CONFIG=NONE DateParserVerbose - Failed to parse timestamp

chillao123 — Tue, 29 Sep 2020 12:41:26 GMT

Hi, I am facing weird issue with timestamp recognition by splunk. Modified timestamp is 2016/11/26 but somehow I see 1998 in splunkd log. File is not getting indexed due to these errors.

Performed the following actions:

Set DATETIME_CONFIG=NONE in forwarder props and indexer props conf file. But I see the following errors:

01-31-2017 19:32:37.365 -0700 WARN DateParserVerbose - A possible timestamp match (Sun Dec 20 20:15:49 1998) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::/tmp/BT99P.BBMXDC48.EXTRACT_161129235057_0643

01-31-2017 19:32:21.236 -0700 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Thu Jan 30 06:07:54 2014). Context: source::/tmp/BT99P.BBMXDC48.EXTRACT_161129235057_0643

Copying below btool output:

Forwarder:

23242 [test_abcd]
23243 ANNOTATE_PUNCT = True
23244 AUTO_KV_JSON = true
23245 BREAK_ONLY_BEFORE =
23246 BREAK_ONLY_BEFORE_DATE = false
23247 CHARSET = UTF-8
23248 DATETIME_CONFIG = NONE
23249 HEADER_MODE =
23250 LEARN_SOURCETYPE = true
23251 LINE_BREAKER_LOOKBEHIND = 100
23252 MAX_DAYS_AGO = 2000
23253 MAX_DAYS_HENCE = 2
23254 MAX_DIFF_SECS_AGO = 3600
23255 MAX_DIFF_SECS_HENCE = 604800
23256 MAX_EVENTS = 256
23257 MAX_TIMESTAMP_LOOKAHEAD = 128
23258 MUST_BREAK_AFTER =
23259 MUST_NOT_BREAK_AFTER =
23260 MUST_NOT_BREAK_BEFORE =
23261 NO_BINARY_CHECK = true
23262 SEGMENTATION = indexing
23263 SEGMENTATION-all = full
23264 SEGMENTATION-inner = inner
23265 SEGMENTATION-outer = outer
23266 SEGMENTATION-raw = none
23267 SEGMENTATION-standard = standard
23268 SHOULD_LINEMERGE = false
23269 TRANSFORMS =
23270 TRUNCATE = 10000
23271 detect_trailing_nulls = false
23272 disabled = false
23273 maxDist = 100
23274 priority =
23275 pulldown_type = true
23276 sourcetype =

Indexer props:
8891 [test_abcd]
8892 ANNOTATE_PUNCT = True
8893 AUTO_KV_JSON = true
8894 BREAK_ONLY_BEFORE =
8895 BREAK_ONLY_BEFORE_DATE = false
8896 CHARSET = UTF-8
8897 DATETIME_CONFIG = NONE
8898 HEADER_MODE =
8899 LEARN_SOURCETYPE = true
8900 LINE_BREAKER_LOOKBEHIND = 100
8901 MAX_DAYS_AGO = 2000
8902 MAX_DAYS_HENCE = 2
8903 MAX_DIFF_SECS_AGO = 3600
8904 MAX_DIFF_SECS_HENCE = 604800
8905 MAX_EVENTS = 256
8906 MAX_TIMESTAMP_LOOKAHEAD = 128
8907 MUST_BREAK_AFTER =
8908 MUST_NOT_BREAK_AFTER =
8909 MUST_NOT_BREAK_BEFORE =
8910 NO_BINARY_CHECK = true
8911 SEGMENTATION = indexing
8912 SEGMENTATION-all = full
8913 SEGMENTATION-inner = inner
8914 SEGMENTATION-outer = outer
8915 SEGMENTATION-raw = none
8916 SEGMENTATION-standard = standard
8917 SHOULD_LINEMERGE = false
8918 TRANSFORMS =
8919 TRUNCATE = 10000
8920 detect_trailing_nulls = false
8921 disabled = false
8922 maxDist = 100
8923 priority =
8924 pulldown_type = true

On OS linux file's timestamp:

File: `BT99P.BBMXDC48.EXTRACT_161129235057_0643'
Size: 18012132 Blocks: 35184 IO Block: 4096 regular file
Device: fd03h/64771d Inode: 524302 Links: 1
Access: (0755/-rwxr-xr-x) Uid: (617339/#####) Gid: (6000000/users)
Access: 2017-01-31 19:31:49.335197997 -0700
Modify: 2016-11-26 00:00:09.000000000 -0700
Change: 2017-01-31 19:14:56.740167230 -0700

Need to load old file with modified timestamp as 2016/11/26. Please advise settings need to be made.

Re: DATETIME_CONFIG=NONE DateParserVerbose - Failed to parse timestamp

gcusello — Wed, 01 Feb 2017 08:27:19 GMT

Hi chillao123,
I don't see in tour props.conf the TIME_FORMAT option that is responsable of the correct timestamp reading.
If you want anhelp to build this option, Could you share an example of your logs?
Bye.
Giuseppe

Re: DATETIME_CONFIG=NONE DateParserVerbose - Failed to parse timestamp

chillao123 — Tue, 29 Sep 2020 12:41:36 GMT

Hi Cusello, I have tab delimited file with 1000 lines and I do not want Splunk to read time from logs. DATTIME_CONFIG is set to

NONE so that it can take file modified timestamp of file in Linux. WIth SHOULD_LINEMERGE set of false, my understanding it that all 1000 lines be converted to 1000 events with file modified timestamp as _time

Re: DATETIME_CONFIG=NONE DateParserVerbose - Failed to parse timestamp

inderjot_rasila — Tue, 29 Sep 2020 12:42:08 GMT

Hi @cussello,
PFB the sample log:
002 T***** DEBITS AIR 17/11/16 XXXXX878*91XX2 XX9987****5280 322555521528000 3704.00 LA4667A 2016-11-25 *07298 QDO RIA/PA** B*A **NCO** AS 110015005 IN-U TE**** DIR GARL* DIST AM07 GLX *RAL XEM 0008 004200 66
002 T***** DEBITS AIR 24/11/16 XXXXX878*1XX2 4O328125 329555583602000 2903.00 LA4667A 2016-11-25 ***07298 R*IR* RNAN/MA BA **NCO** AS 110015005 IN-U TE**** DIR GARL* DIST AM07 GLX *RAL XEM 0001 004233

Re: DATETIME_CONFIG=NONE DateParserVerbose - Failed to parse timestamp

woodcock — Thu, 02 Mar 2017 22:33:29 GMT

I agree that your configurations look acceptable but check out this Q&A:
https://answers.splunk.com/answers/455406/why-am-i-getting-dateparserverbose-warnings-althou.html

According to that, you need to remove the DATETIME_CONFIG=NONE from your indexers, which is exactly what I would try. If this fixes it, though, this really should be reported as a bug because it means that a setting that should AT MOST cause the Indexers NOT to do any timestamping, actually turns this back on.