topic Re: Extract timestamp in Epoch (milliseconds) to date in Getting Data In

Extract timestamp in Epoch (milliseconds) to date

hiwell — Sat, 26 Jun 2010 05:43:30 GMT

Hello,

I am currently extracting a field from some event which looks like

Start_Time_ms=1277221722297

My event has no timestamp and this is the only time-related data I have, so I am trying to convert this to a readable date format (ie mm/dd/yy hh:mm:ss). I noticed that people have asked similar questions but all the answers seems to suggest to do this within the search command. However I was hoping to manipulate this number in a config file so that I would have a timestamp for my events. Are these types of operations possible? Or do I need to pre-process the data (again :< )? It would be cool if Splunk supported converting Epoch -> to date.

well thanks in advance.

Re: Extract timestamp in Epoch (milliseconds) to date

jrodman — Sat, 26 Jun 2010 05:46:00 GMT

Is this "How can i get splunk to identify the time of an event by a timestamp in seconds from epoch?"

If that's the question, the answer is

TIME_FORMAT=%s
TIME_PREFIX=Start_Time_ms=

Err update: what I gave will pull seconds from epoch, not milliseconds. I don't know if our extractor will give the right answer for milliseconds. It might just pull the first n digits, but could be off for very old timestamps. I haven't tested.

Re: Extract timestamp in Epoch (milliseconds) to date

gkanapathy — Sat, 26 Jun 2010 05:54:30 GMT

More detail here: http://www.splunk.com/base/Documentation/latest/Admin/Configuretimestamprecognition#Configure_timestamps

Re: Extract timestamp in Epoch (milliseconds) to date

twinspop — Tue, 29 Jun 2010 04:17:11 GMT

Splunk recognizes epoch time in my tests. Now the milliseconds piece is tougher. Maybe passing through sed at index time?

[wanky_logs]
SEDCMD-inseconds = s/Start_Time_ms=(\d{10})(\d{3})/\1.\2/

If the timestamp isn't the first thing in the log file, you'll need to specify the TIME_PREFIX possibly.

Re: Extract timestamp in Epoch (milliseconds) to date

twinspop — Tue, 29 Jun 2010 04:55:11 GMT

Sorry, timestamp extraction is before the SEDCMD fires, so this won't work.

Re: Extract timestamp in Epoch (milliseconds) to date

stanwin — Mon, 04 Apr 2016 14:05:48 GMT

Well SPLUNK (v 6.2.3 ) with automatic timestamp recognition parses the timestamp ( epoch in milliseconds), but there is no strptime equivalent for that so I cant specify custom timestamp extraction.

The current version %s supports Epoch with 10 digits only.

Using %s to parse the epoch time ( in miliseconds) gives a gibberish date.. Its just hit the max of 10 digit epoch i think.

 epoch:  1459495744806
     extracted timestamp: 31/12/9999 23:59:59.999

So if I have a epoch in milliseconds appears automatic timestamp extraction is the only way as of now than?

here is the difference between them:

    1459495744806
    GMT: Fri, 01 Apr 2016 07:29:04.806 GMT

    1459495744
    GMT: Fri, 01 Apr 2016 07:29:04 GMT

Re: Extract timestamp in Epoch (milliseconds) to date

stanwin — Mon, 04 Apr 2016 16:18:41 GMT

Got it was simple enough!

The fact that it was one long integer dissuaded me from using %3N in combination to %s. But it works !

Below will parse timestamps with epoch time in milliseconds e.e. 1459495744806

TIME_FORMAT=%s%3N

Re: Extract timestamp in Epoch (milliseconds) to date

drutstein — Thu, 15 Jun 2017 07:16:00 GMT

I downvoted this post because this isn't the correct answer for milliseconds as the question asked.
The documentation in Splunk documentation for timestamp recognition identifies that you can use either
TIME_FORMAT=%s%3N
or
TIME_FORMAT=%s%Q

Re: Extract timestamp in Epoch (milliseconds) to date

morethanyell — Fri, 07 Jun 2019 03:19:58 GMT

I downvoted this post because %s is not the right format for milliseconds. this will cause you errors