i have the following problem:
I have to read in logfiles with Splunk that contain an uncommon timestamp format. After a little bit of research, i realized that the timestamps are in a modified version of the windows FILETIME format. The definition is basically :
a 64-bit value representing the number of 100-nanosecond intervals since January 1, 1601 (UTC).
As far as i can see, Splunk is not able to correctly parse this timestamp format. Additionally, there is the small modification, that i mentioned before. In my files the last four digits of the timestamp are cut.
Here a small example:
So if i am correct, this should be something like "100-microsecond intervals since January 1, 1601 (UTC)"
I know i could just read it in and do some math to convert the timestamp to epoch, but i would prefer to get a clean timestamp recognition working at indextime.
Is there any way splunk can correctly recognize this timestampformat, or at least the original FILETIME format?
Note, this timestamp doesn't map to May 2011. In fact it maps to Thursday, May 1, 2014 7:59:52pm.
Mapping to May 2011 is what Splunk does by default, incorrectly interpret this as a unix epoch timestamp with extra precision.
Manual conversion tool: http://www.silisoftware.com/tools/date.php
Are you sure that it works ?
I thought that the timestamp detection was happening before the transforms regex replacement.
I haven't tried this kind of conversions, but if i have to put my 2 cents in...
I used http://www.epochconverter.com/ to check the sample timestamps that you have given. I feel the second one has two zeroes extra.
Both 1304346239 and 1304346239155700 resolve to same date & time. That is GMT: Mon, 02 May 2011 14:23:59 GMT
So, i feel you need to ignore last four digits of timestamps that are present in your files.
As Yann pointed out use TIME_FORMAT in props.conf
for the Windows 1601 epoch timestamp, I do not know if the format can be detected at indextime.
Here is a manual way to convert a search time http://answers.splunk.com/answers/115016/windows-filetime-timestamp-to-human-readable
For the "130434623915570000 " 1970 epoch timestamp with milliseconds, you can try "%S%4N" for timestamp detection.
Hi, thanks for your answer. I already did the convertion of the timestamp in splunk, but as mentioned i would prefer to correctly detect the timestamp at index time.
Also the "130434623915570000" timestamp is no 1970 epoch timestamp with higher precision, it is still filetime. With your approach, i would gather the same result as already mentioned by @strive.