Getting Data In

DATETIME_CONFIG=NONE DateParserVerbose - Failed to parse timestamp

Explorer

Hi, I am facing weird issue with timestamp recognition by splunk. Modified timestamp is 2016/11/26 but somehow I see 1998 in splunkd log. File is not getting indexed due to these errors.

Performed the following actions:

Set DATETIME_CONFIG=NONE in forwarder props and indexer props conf file. But I see the following errors:

01-31-2017 19:32:37.365 -0700 WARN DateParserVerbose - A possible timestamp match (Sun Dec 20 20:15:49 1998) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAXDAYSAGO and MAXDAYSHENCE. Context: source::/tmp/BT99P.BBMXDC48.EXTRACT1611292350570643

01-31-2017 19:32:21.236 -0700 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Thu Jan 30 06:07:54 2014). Context: source::/tmp/BT99P.BBMXDC48.EXTRACT1611292350570643

Copying below btool output:

Forwarder:

23242 [testabcd]
23243 ANNOTATE
PUNCT = True
23244 AUTOKVJSON = true
23245 BREAKONLYBEFORE =
23246 BREAKONLYBEFOREDATE = false
23247 CHARSET = UTF-8
23248 DATETIME
CONFIG = NONE
23249 HEADERMODE =
23250 LEARN
SOURCETYPE = true
23251 LINEBREAKERLOOKBEHIND = 100
23252 MAXDAYSAGO = 2000
23253 MAXDAYSHENCE = 2
23254 MAXDIFFSECSAGO = 3600
23255 MAX
DIFFSECSHENCE = 604800
23256 MAXEVENTS = 256
23257 MAX
TIMESTAMPLOOKAHEAD = 128
23258 MUST
BREAKAFTER =
23259 MUST
NOTBREAKAFTER =
23260 MUSTNOTBREAKBEFORE =
23261 NO
BINARYCHECK = true
23262 SEGMENTATION = indexing
23263 SEGMENTATION-all = full
23264 SEGMENTATION-inner = inner
23265 SEGMENTATION-outer = outer
23266 SEGMENTATION-raw = none
23267 SEGMENTATION-standard = standard
23268 SHOULD
LINEMERGE = false
23269 TRANSFORMS =
23270 TRUNCATE = 10000
23271 detecttrailingnulls = false
23272 disabled = false
23273 maxDist = 100
23274 priority =
23275 pulldown_type = true
23276 sourcetype =

Indexer props:
8891 [testabcd]
8892 ANNOTATE
PUNCT = True
8893 AUTOKVJSON = true
8894 BREAKONLYBEFORE =
8895 BREAKONLYBEFOREDATE = false
8896 CHARSET = UTF-8
8897 DATETIME
CONFIG = NONE
8898 HEADERMODE =
8899 LEARN
SOURCETYPE = true
8900 LINEBREAKERLOOKBEHIND = 100
8901 MAXDAYSAGO = 2000
8902 MAXDAYSHENCE = 2
8903 MAXDIFFSECSAGO = 3600
8904 MAX
DIFFSECSHENCE = 604800
8905 MAXEVENTS = 256
8906 MAX
TIMESTAMPLOOKAHEAD = 128
8907 MUST
BREAKAFTER =
8908 MUST
NOTBREAKAFTER =
8909 MUSTNOTBREAKBEFORE =
8910 NO
BINARYCHECK = true
8911 SEGMENTATION = indexing
8912 SEGMENTATION-all = full
8913 SEGMENTATION-inner = inner
8914 SEGMENTATION-outer = outer
8915 SEGMENTATION-raw = none
8916 SEGMENTATION-standard = standard
8917 SHOULD
LINEMERGE = false
8918 TRANSFORMS =
8919 TRUNCATE = 10000
8920 detecttrailingnulls = false
8921 disabled = false
8922 maxDist = 100
8923 priority =
8924 pulldown_type = true

On OS linux file's timestamp:

File: `BT99P.BBMXDC48.EXTRACT1611292350570643'
Size: 18012132 Blocks: 35184 IO Block: 4096 regular file
Device: fd03h/64771d Inode: 524302 Links: 1
Access: (0755/-rwxr-xr-x) Uid: (617339/#####) Gid: (6000000/users)
Access: 2017-01-31 19:31:49.335197997 -0700
Modify: 2016-11-26 00:00:09.000000000 -0700
Change: 2017-01-31 19:14:56.740167230 -0700

Need to load old file with modified timestamp as 2016/11/26. Please advise settings need to be made.

0 Karma

Esteemed Legend

I agree that your configurations look acceptable but check out this Q&A:
https://answers.splunk.com/answers/455406/why-am-i-getting-dateparserverbose-warnings-althou.html

According to that, you need to remove the DATETIME_CONFIG=NONE from your indexers, which is exactly what I would try. If this fixes it, though, this really should be reported as a bug because it means that a setting that should AT MOST cause the Indexers NOT to do any timestamping, actually turns this back on.

0 Karma

Legend

Hi chillao123,
I don't see in tour props.conf the TIME_FORMAT option that is responsable of the correct timestamp reading.
If you want anhelp to build this option, Could you share an example of your logs?
Bye.
Giuseppe

0 Karma

Hi @cussello,
PFB the sample log:
002 T***** DEBITS AIR 17/11/16 XXXXX87891XX2 XX9987***5280 322555521528000 3704.00 LA4667A 2016-11-25 *07298 QDO RIA/PA** BA **NCO* AS 110015005 IN-U TE**** DIR GARL* DIST AM07 GLX *RAL XEM 0008 004200 66
002 T***** DEBITS AIR 24/11/16 XXXXX8781XX2 4O3281**25 329555583602000 2903.00 LA4667A 2016-11-25 *07298 RIR* RNAN/MA BA *NCO*** AS 110015005 IN-U TE**** DIR GARL* DIST AM07 GLX *RAL XEM 0001 004233

0 Karma

Explorer

Hi Cusello, I have tab delimited file with 1000 lines and I do not want Splunk to read time from logs. DATTIMECONFIG is set to

NONE so that it can take file modified timestamp of file in Linux. WIth SHOULD
LINEMERGE set of false, my understanding it that all 1000 lines be converted to 1000 events with file modified timestamp as _time

0 Karma