Getting Data In

Assign sequential timestamps to data on data load

NK_1
Path Finder

I tried loading a textfile (via "splunk add oneshot datafile.txt") containing just IP addresses into Splunk v5.0, and Splunk tried to assign some really strange timestamps (2010 - 2019?) to some of those IP Address lines.

I suspect it tried to interpret some of the the ip address numbers as dates:


$ grep DateParserVerbose ./var/log/splunk/splunkd.log | grep 2019

07-16-2013 17:31:16.085 -0700 WARN DateParserVerbose - Accepted time (Sun Nov 24 19:11:24 2019) is suspiciously far away from the previous event's time (Sun Nov 21 12:33:24 2010), but still accepted because it was extracted by the same pattern. Context: source::(filename.csv)|host::(hostname)|(filename)|

07-16-2013 17:31:16.085 -0700 WARN DateParserVerbose - A possible timestamp match (Sun Nov 24 19:11:24 2019) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::(filename.csv)|host::(hostname)|(filename)|

Is there a way to make Splunk assign sequential timestamps and not try to parse a timestamp from the data?

Tags (3)
0 Karma

wagnerbianchi
Splunk Employee
Splunk Employee

If Splunk does not find a timestamp for each line contained into a file, the timestamp of the file creation will be used asvthe event timestamp. What is the timestamp format of your operation system? Tks!

0 Karma

NK_1
Path Finder

I don't recall specifying or overriding any default for the timestamp format, but here's some info:


$ echo $OSTYPE
linux-gnu

$ date
Mon Jul 22 15:01:50 PDT 2013

0 Karma

NK_1
Path Finder

I ended up doing this to prevent Splunk from guessing timestamps for now, but I'm looking for a better solution:

# add timestamp and key to every line in text file 
cat datafile.txt | xargs -d"\n" -I {} date +"%Y.%m.%d %H:%M:%S.%N ip={}" > datafile1.txt

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

From Data to Insight: Announcing the Winners of the Splunk Dashboard Contest

Hi Splunkers, First off, thank you to everyone who participated in our very first From Data to Insight: The ...

Splunk Developers: Construct Your Future at the .conf26 Builder Bar

Calling all Splunk architects, platform admins, and app developers: the site is open, and the blueprints are ...

Quick connection discovery mode for forwarders

When a Splunk forwarder loses connectivity to its indexers, it does not always reconnect immediately. In many ...