Getting Data In

Assign sequential timestamps to data on data load

NK_1
Path Finder

I tried loading a textfile (via "splunk add oneshot datafile.txt") containing just IP addresses into Splunk v5.0, and Splunk tried to assign some really strange timestamps (2010 - 2019?) to some of those IP Address lines.

I suspect it tried to interpret some of the the ip address numbers as dates:


$ grep DateParserVerbose ./var/log/splunk/splunkd.log | grep 2019

07-16-2013 17:31:16.085 -0700 WARN DateParserVerbose - Accepted time (Sun Nov 24 19:11:24 2019) is suspiciously far away from the previous event's time (Sun Nov 21 12:33:24 2010), but still accepted because it was extracted by the same pattern. Context: source::(filename.csv)|host::(hostname)|(filename)|

07-16-2013 17:31:16.085 -0700 WARN DateParserVerbose - A possible timestamp match (Sun Nov 24 19:11:24 2019) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::(filename.csv)|host::(hostname)|(filename)|

Is there a way to make Splunk assign sequential timestamps and not try to parse a timestamp from the data?

Tags (3)
0 Karma

wagnerbianchi
Splunk Employee
Splunk Employee

If Splunk does not find a timestamp for each line contained into a file, the timestamp of the file creation will be used asvthe event timestamp. What is the timestamp format of your operation system? Tks!

0 Karma

NK_1
Path Finder

I don't recall specifying or overriding any default for the timestamp format, but here's some info:


$ echo $OSTYPE
linux-gnu

$ date
Mon Jul 22 15:01:50 PDT 2013

0 Karma

NK_1
Path Finder

I ended up doing this to prevent Splunk from guessing timestamps for now, but I'm looking for a better solution:

# add timestamp and key to every line in text file 
cat datafile.txt | xargs -d"\n" -I {} date +"%Y.%m.%d %H:%M:%S.%N ip={}" > datafile1.txt

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

What Is Splunk? Here’s What You Can Do with Splunk

Hey Splunk Community, we know you know Splunk. You likely leverage its unparalleled ability to ingest, index, ...

Level Up Your .conf25: Splunk Arcade Comes to Boston

With .conf25 right around the corner in Boston, there’s a lot to look forward to — inspiring keynotes, ...

Manual Instrumentation with Splunk Observability Cloud: How to Instrument Frontend ...

Although it might seem daunting, as we’ve seen in this series, manual instrumentation can be straightforward ...