topic Re: Assign sequential timestamps to data on data load in Getting Data In

Assign sequential timestamps to data on data load

NK_1 — Mon, 28 Sep 2020 14:23:54 GMT

I tried loading a textfile (via "splunk add oneshot datafile.txt") containing just IP addresses into Splunk v5.0, and Splunk tried to assign some really strange timestamps (2010 - 2019?) to some of those IP Address lines.

I suspect it tried to interpret some of the the ip address numbers as dates:



$ grep DateParserVerbose ./var/log/splunk/splunkd.log | grep 2019

07-16-2013 17:31:16.085 -0700 WARN DateParserVerbose - Accepted time (Sun Nov 24 19:11:24 2019) is suspiciously far away from the previous event's time (Sun Nov 21 12:33:24 2010), but still accepted because it was extracted by the same pattern. Context: source::(filename.csv)|host::(hostname)|(filename)|

07-16-2013 17:31:16.085 -0700 WARN DateParserVerbose - A possible timestamp match (Sun Nov 24 19:11:24 2019) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::(filename.csv)|host::(hostname)|(filename)|

Is there a way to make Splunk assign sequential timestamps and not try to parse a timestamp from the data?

Re: Assign sequential timestamps to data on data load

NK_1 — Sat, 20 Jul 2013 02:22:00 GMT

I ended up doing this to prevent Splunk from guessing timestamps for now, but I'm looking for a better solution:

# add timestamp and key to every line in text file 

cat datafile.txt | xargs -d"\n" -I {} date +"%Y.%m.%d %H:%M:%S.%N ip={}" > datafile1.txt

Re: Assign sequential timestamps to data on data load

wagnerbianchi — Sat, 20 Jul 2013 03:08:01 GMT

If Splunk does not find a timestamp for each line contained into a file, the timestamp of the file creation will be used asvthe event timestamp. What is the timestamp format of your operation system? Tks!

Re: Assign sequential timestamps to data on data load

NK_1 — Mon, 22 Jul 2013 22:05:48 GMT

I don't recall specifying or overriding any default for the timestamp format, but here's some info:



$ echo $OSTYPE

linux-gnu

$ date
Mon Jul 22 15:01:50 PDT 2013