5

Note: I think datetime64 is doing the right thing. So I'll just leave the post up in case it's useful.

As of numpy 1.7.0, seconds passed in to a np.datetime64 are interpreted as being in a local timezone. Is there a clean and fast way to import a unix utc seconds to np.datetime64? I've got arrays with 50M of these and it seems that there should be a way to tell np.datetime64 that my seconds value is UTC, no?

datetime.datetime.utcfromtimestamp(1338624706)
datetime.datetime(2012, 6, 2, 8, 11, 46)  # this is the time I'm looking for

np.datetime64(1338624706, 's')
numpy.datetime64('2012-06-02T01:11:46-0700')  # Darn you ISO!  Off by 7 hours

dt64 = np.datetime64(1338624706, 's')
dt64.astype(datetime.datetime)
datetime.datetime(2012, 6, 2, 8, 11, 46)  # Wait, did it do the right thing?

# This seems like the best option at the moment,
# but requires building datetime.datetime objects:
dt64 = np.datetime64(datetime.datetime.utcfromtimestamp(1338624706))
numpy.datetime64('2012-06-02T01:11:46.000000-0700') # Show this
dt64.astype(datetime.datetime)
datetime.datetime(2012, 6, 2, 8, 11, 46)  # Looks like it worked

I really do not want to resort to string operations. I would be nice to be able to convert an array of unix utc ints or floats straight to the correct dt64.

https://stackoverflow.com/a/13704307/417578 implies that numpy 1.8.0 might do what I want, but is there something that will work in 1.7.0?

Improve this question
2
  • 1
    "So I'll just leave the post up in case it's useful." --- you should post the solution ('best option' I guess) as an answer so it shows up as answered. Commented Feb 24, 2013 at 16:54
  • possible duplicate of Converting between datetime, Timestamp and datetime64 Commented Feb 24, 2013 at 18:57

2 Answers 2

Reset to default
4

Heres another way in pandas (which handles the quirks in different versions of numpy datetime64 correctly, so this works in numpy 1.6.2) - I think u might need current master for this (0.11-dev)

# obviously replace this by your utc seconds
# need to convert to the default in pandas of datetime64[ns]
z = pd.Series([(1338624706 + i)*1e9 for i in range(50)],dtype='datetime64[ns]')

In [35]: z.head()
Out[35]: 
0   2012-06-02 08:11:46
1   2012-06-02 08:11:47
2   2012-06-02 08:11:48
3   2012-06-02 08:11:49
4   2012-06-02 08:11:50
Dtype: datetime64[ns]

# turn it into a DatetimeIndex and localize
lidx = pd.DatetimeIndex(z).tz_localize('UTC')

<class 'pandas.tseries.index.DatetimeIndex'>
[2012-06-02 08:11:46, ..., 2012-06-02 08:12:35]
Length: 50, Freq: None, Timezone: UTC

# now you have a nice object to say convert timezones
In [44]: lidx.tz_convert('US/Eastern')
Out[44]: 
<class 'pandas.tseries.index.DatetimeIndex'>
[2012-06-02 04:11:46, ..., 2012-06-02 04:12:35]
Length: 50, Freq: None, Timezone: US/Eastern
Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

2

Perhaps I misunderstand the question, but isn't the timezone just a display issue?

utc_time = datetime.datetime.utcnow()
print utc_time
dt64 =  np.datetime64(utc_time)
print dt64
print dt64.astype(datetime.datetime)


2013-02-24 17:30:53.586297
2013-02-24T11:30:53.586297-0600
2013-02-24 17:30:53.586297

The time hasn't been 'changed' in any way:

some_time = datetime.datetime.utcfromtimestamp(1338624706)
dt64 = np.datetime64(1338624706,'s')
print dt64.astype(int64)
1338624706

This is as of numpy 1.7.

Improve this answer

1 Comment

Yes, I was just miss-reading the output. My off by 7 comment is on a line that is showing "-0700". Setting the timezone makes life easier. e.g. os.environ['TZ'] = 'UTC'

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.