topic Re: Timestamp sometimes not recognized in batch in Getting Data In

Timestamp sometimes not recognized in batch

Bradysmall — Mon, 28 Sep 2020 13:03:37 GMT

I have files that are simple CSV.. using a batch to get them into splunk.

Logs look like this..

12-06-2012 23:58:53.738,JBSP_P03,2_SPORTS/WIDEVID,CAM_05,Take,J_LIGHT2,OK

in my props, I have

MAX_TIMESTAMP_LOOKAHEAD = 23
SHOULD_LINEMERGE = false
BREAK_ONLY_BEFORE_DATE = true
LINE_BREAKER = ([\r\n]+)
TRUNCATE = 999999
KV_MODE = None
TIME_FORMAT = %m-%d-%Y %H:%M:%S.%N

50% of the events get timestamped correctly.

The other 50% get stamped for when the file got copied into the batch folder..

12-06-2012 23:58:53.738,JBSP_P03,2_SPORTS/WIDEVID,CAM_05,Take,J_LIGHT2,OK - Stamped in splunk correctly.

10-08-2004 23:07:27.156,TGQC32N,3_NON/AES1/2,PRFD1912,Take Path,8191,OK - Stamped incorrectly in splunk.

Re: Timestamp sometimes not recognized in batch

emiller42 — Mon, 07 Jan 2013 19:20:05 GMT

You may need to add a MAX_DAYS_AGO line to your props.conf stanza. By default, Splunk will throw out any extracted timestamps more than 2000 days old. Your example is past that threshold, which is why it used index time.

from the props.conf spec

MAX_DAYS_AGO = integer
* Specifies the maximum number of days past, from the current date, that an extracted date can be valid.
* For example, if MAX_DAYS_AGO = 10, Splunk ignores dates that are older than 10 days ago.
* Defaults to 2000 (days), maximum 10951.
* IMPORTANT: If your data is older than 2000 days, increase this setting.

Re: Timestamp sometimes not recognized in batch

Bradysmall — Mon, 07 Jan 2013 20:07:28 GMT

Thanks. I stumbled into something similar and was just in the middle of changing that variable to check it on myself and it worked.

Re: Timestamp sometimes not recognized in batch

Bradysmall — Mon, 07 Jan 2013 20:07:48 GMT

Thanks. I stumbled into something similar and was just in the middle of changing that variable to check it on myself and it worked.