Getting Data In

Assign sequential timestamps to data on data load

NK_1
Path Finder

I tried loading a textfile (via "splunk add oneshot datafile.txt") containing just IP addresses into Splunk v5.0, and Splunk tried to assign some really strange timestamps (2010 - 2019?) to some of those IP Address lines.

I suspect it tried to interpret some of the the ip address numbers as dates:


$ grep DateParserVerbose ./var/log/splunk/splunkd.log | grep 2019

07-16-2013 17:31:16.085 -0700 WARN DateParserVerbose - Accepted time (Sun Nov 24 19:11:24 2019) is suspiciously far away from the previous event's time (Sun Nov 21 12:33:24 2010), but still accepted because it was extracted by the same pattern. Context: source::(filename.csv)|host::(hostname)|(filename)|

07-16-2013 17:31:16.085 -0700 WARN DateParserVerbose - A possible timestamp match (Sun Nov 24 19:11:24 2019) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::(filename.csv)|host::(hostname)|(filename)|

Is there a way to make Splunk assign sequential timestamps and not try to parse a timestamp from the data?

Tags (3)
0 Karma

wagnerbianchi
Splunk Employee
Splunk Employee

If Splunk does not find a timestamp for each line contained into a file, the timestamp of the file creation will be used asvthe event timestamp. What is the timestamp format of your operation system? Tks!

0 Karma

NK_1
Path Finder

I don't recall specifying or overriding any default for the timestamp format, but here's some info:


$ echo $OSTYPE
linux-gnu

$ date
Mon Jul 22 15:01:50 PDT 2013

0 Karma

NK_1
Path Finder

I ended up doing this to prevent Splunk from guessing timestamps for now, but I'm looking for a better solution:

# add timestamp and key to every line in text file 
cat datafile.txt | xargs -d"\n" -I {} date +"%Y.%m.%d %H:%M:%S.%N ip={}" > datafile1.txt

0 Karma
Get Updates on the Splunk Community!

Enter the Agentic Era with Splunk AI Assistant for SPL 1.4

  🚀 Your data just got a serious AI upgrade — are you ready? Say hello to the Agentic Era with the ...

Stronger Security with Federated Search for S3, GCP SQL & Australian Threat ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Accelerating Observability as Code with the Splunk AI Assistant

We’ve seen in previous posts what Observability as Code (OaC) is and how it’s now essential for managing ...