i have the following problem:
I have to read in logfiles with Splunk that contain an uncommon timestamp format. After a little bit of research, i realized that the timestamps are in a modified version of the windows FILETIME format. The definition is basically :
a 64-bit value representing the number of 100-nanosecond intervals since January 1, 1601 (UTC).
As far as i can see, Splunk is not able to correctly parse this timestamp format. Additionally, there is the small modification, that i mentioned before. In my files the last four digits of the timestamp are cut.
Here a small example:
So if i am correct, this should be something like "100-microsecond intervals since January 1, 1601 (UTC)"
I know i could just read it in and do some math to convert the timestamp to epoch, but i would prefer to get a clean timestamp recognition working at indextime.
Is there any way splunk can correctly recognize this timestampformat, or at least the original FILETIME format?
for the Windows 1601 epoch timestamp, I do not know if the format can be detected at indextime.
Here is a manual way to convert a search time http://answers.splunk.com/answers/115016/windows-filetime-timestamp-to-human-readable
For the "130434623915570000 " 1970 epoch timestamp with milliseconds, you can try "%S%4N" for timestamp detection.
Hi, thanks for your answer. I already did the convertion of the timestamp in splunk, but as mentioned i would prefer to correctly detect the timestamp at index time.
Also the "130434623915570000" timestamp is no 1970 epoch timestamp with higher precision, it is still filetime. With your approach, i would gather the same result as already mentioned by @strive.
I haven't tried this kind of conversions, but if i have to put my 2 cents in...
I used http://www.epochconverter.com/ to check the sample timestamps that you have given. I feel the second one has two zeroes extra.
Both 1304346239 and 1304346239155700 resolve to same date & time. That is GMT: Mon, 02 May 2011 14:23:59 GMT
So, i feel you need to ignore last four digits of timestamps that are present in your files.
As Yann pointed out use TIME_FORMAT in props.conf
Are you sure that it works ?
I thought that the timestamp detection was happening before the transforms regex replacement.
Note, this timestamp doesn't map to May 2011. In fact it maps to Thursday, May 1, 2014 7:59:52pm.
Mapping to May 2011 is what Splunk does by default, incorrectly interpret this as a unix epoch timestamp with extra precision.
Manual conversion tool: http://www.silisoftware.com/tools/date.php
I used the manual conversion tool that you specified.
For 130434623915570000 if the filetime is chosen as Input format then as you say it displays Thursday, May 1, 2014 7:59:52pm