Getting Data In

Can Splunk strptime() work with the date before 1970-01-01 in epoch format?

luxiaobin
Explorer

Sometime I have a timestamp like -633945600.000 in my data. I found a previous post where someone said Splunk only supports events with an epoch time greater than zero. http://answers.splunk.com/answers/8038/will-splunk-index-events-older-than-1970-1-1.html
What can I do to change the format for a timestamp like this? Please help!

thanks!

0 Karma

MuS
SplunkTrust
SplunkTrust

Okay even the Splunk Master Mind Stephen Sorkin has answered that way back in time, here are some simple test to show that it is not possible either using a negative epochtime nor with a string.

this will work:

index=_internal | head 1 | eval foo="386380800" | eval baz=strftime(foo, "%F %T") | table foo baz

because it is using a valid epoch time and the strftime().

this will not work neither for strftime nor strptime, because of the negative epochtime used:

index=_internal | head 1 | eval foo="-386380800" | eval baz=strftime(foo, "%F %T") | table foo baz
index=_internal | head 1 | eval baz=strptime("-386380800", "%F %T") | table baz

cheers, MuS

markthompson
Builder

To be honest, Epoch itself by definition is the time in seconds since 1st january 1970.... So how can it be a valid epoch time if it's negative value?

I don't think it would actually work.

jrodman
Splunk Employee
Splunk Employee

We use a couple of internal representations, but one of them is time_t, which is not guaranteed to be, but on all systems we run on is an unsigned value after 1970. So yes this is a no-go unless you go to a lot of trouble to represent your time values in some other way that obviously won't have full featured support.

0 Karma

luxiaobin
Explorer

the strptime() cant work with date before 1970, not only epoch time but the format like 1969-01-01.
but in my system,the date is the user
s date of borth, I need got the user`s age .so I change the time format from epoch to %Y-%m-%d use python in the csv file,and use substr() to get the born year in search string.

0 Karma

markthompson
Builder

Hi Luxiaobin, please mark this as an answer if it's the correct comments. It appears that I was correct and if you're going to be storing values as times, I'd be tempted to say do it as different fields, dob_day, dob_month, dob_year.

Something along those lines

0 Karma

luxiaobin
Explorer

I use python write a script change the time format
the strptime() cant work with date before 1970, not only epoch time but the format like 1969-01-01.
but in my system,the date is the user
s date of borth, I need got the user`s age .so I change the time format from epoch to %Y-%m-%d use python in the csv file,and use substr() to get the born year in search string.
Is this a good method?

0 Karma

MuS
SplunkTrust
SplunkTrust

Hi luxiaobin,

Take a look at the docs about how timestamp assignment works http://docs.splunk.com/Documentation/Splunk/6.2.1/Data/HowSplunkextractstimestamps

Hope this helps ...

cheers, MuS

luxiaobin
Explorer

ps I got the data from the csv files ,if I want change the time format,how can I do?

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!