topic Re: Problems with timestamp detection with log2timeline output in Getting Data In

Problems with timestamp detection with log2timeline output

mpo — Tue, 20 May 2014 10:57:51 GMT

Hi there,

I'm trying to import a log2timeline output (csv) into splunk, but timestamp detection fails, when I try to define a new sourcetype.

The time information is located at the line start an in the following format:
01/01/1970,00:00:00,UTC,...

I've already set the format for strptime to %m/%d/%Y,%H:%M:%S an tried automatic timestamp search as well as defining a regex.
But the only string which is highlighted (and detected as potential timestamp) ist 00:00:00.

Ipython tells that the format sting matches the timestamp:

In [11]: time.strptime("01/01/1970,00:00:00","%m/%d/%Y,%H:%M:%S") Out[11]: time.struct_time(tm_year=1970, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=3, tm_yday=1, tm_isdst=-1)

I also tried the following things:

Removing the "," between date and time
Adding a prefix to each line
Changing the dates to a day later than 01/01/1970
Swapping %m and %d just in case of any doubts

Installed Splunk version is 6.1.1

Does anybody have an idea what I can do that splunk correctly recognizes the time of the lines?
Kind regards in advance!
Markus

First lines of my log are:

01/01/1970,00:00:00,UTC,..C.,REG,NTUSER key : Typed Paths,Last Written,-,xxx,[\Software\Microsoft\Windows\CurrentVersion\Explorer\TypedPaths]... 01/01/1970,00:00:00,UTC,..C.,REG,NTUSER key : RDP Connection,Last Written,-,xxx,[\Software\Microsoft\Terminal Server Client\Default] MRU1:... 01/01/1970,00:00:00,UTC,..C.,REG,NTUSER key : Typed Paths,Last Written,-,xxx,[\Software\Microsoft\Windows\CurrentVersion\Explorer\TypedPaths] ...

Re: Problems with timestamp detection with log2timeline output

jeremiahc4 — Mon, 28 Sep 2020 16:39:58 GMT

If those dates are your literal dates, they are beyond the maximum number of days back Splunk will recognize (MAX_DAYS_AGO). You need to modify that max in props.conf in order to catch something that far back.

According to the documentation the maximum this number can be is 10951 though, I'm thinking that will only get you back around 1984.

http://docs.splunk.com/Documentation/Splunk/6.1.1/Admin/Propsconf

Re: Problems with timestamp detection with log2timeline output

mpo — Mon, 28 Sep 2020 16:40:01 GMT

Weird!
Not quite a new issue - but I never thought, that the MAY_DAYS_AGO setting might be there and causes the trouble.

Thanks a lot!

Is there any additional attribute for props.conf, which sets a custom fallback time, when no timestamp is detected? I don't find any in the documentation and in answers.splunk.com.

Re: Problems with timestamp detection with log2timeline output

jeremiahc4 — Tue, 20 May 2014 12:42:39 GMT

It looks like DATETIME_CONFIG could be used to mess with what gets used in the case when Splunk doesn't recognize a date. . It appears to point to a datetime.xml file which is used to recognize your timestamp. Perhaps if you're XML savvy, you could make mods there. I've not played with this before though, so can't recommend anything with it. As with any edits, make a backup copy first!