New style python buffer protocol and numpy arrays

Question

I'm trying to write a fast non copy interface for my python binding of a commercial image processing library. I implemented the new-style buffer api protocol which looks ok according to memoryview():

import hirsch as H
import numpy as np

w,h = 7,5
img = H.HImage.GenImageConst('byte',w,h)
m = memoryview(img)
print 'ndim shape=',m.ndim,m.shape
# -> 2 (5L, 7L)

What I don't understand is why numpy doesn't catch this interface?

a = np.array(img)
print 'ndim size shape=',a.ndim,a.size,a.shape
# -> 0 1 ()

Am I doing something wrong, or should I just resort to using the numpy array interface, which works, though it copies the data?

Note that I'm using python 2.7

It may be that there's something surprising going on --- but it's difficult to say what without knowing what is in your implementation of bf_getbuffer and bf_releasebuffer. The new buffer interface is supposed to work also on Python 2.x (indeed: np.array(memoryview('asd')) == np.array([ 97, 115, 100], dtype=np.uint8). (Note also that the Numpy array interface does not need to copy the data.) — pv.
– pv., Commented Aug 17, 2014 at 14:03

Bakuriu · Accepted Answer · 2014-08-17 07:40:28Z

3

The np.array function expects an array-like object, not a buffer:

array(object, dtype=None, copy=True, order=None, subok=False, ndmin=0)

Create an array.

object : array_like
An array, any object exposing the array interface, an object whose __array__ method returns an array, or any (nested) sequence.

If you want to create an array from a buffer you should use np.frombuffer:

frombuffer(buffer, dtype=float, count=-1, offset=0)

Interpret a buffer as a 1-dimensional array.

buffer : buffer_like
An object that exposes the buffer interface.

Currently your object, as far as numpy is concerned, is a scalar:

In [7]: a=np.array(1)

In [8]: a.ndim,a.size,a.shape
Out[8]: (0, 1, ())

edited Aug 17, 2014 at 7:40

answered Aug 17, 2014 at 7:34

Bakuriu

103k23 gold badges209 silver badges236 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Dov Grobgeld Over a year ago

If the new buffer is the standard python interface, I'm surprised that np array does not support it! I figured out that the following line can be used for the conversion: a = np.frombuffer(m.tobytes(),{'i8':'i1'}[m.format]).reshape(m.shape). But it also involves copying in m.tobytes().

pv. Over a year ago

The new buffer interface is supported also in np.array: np.array(memoryview(np.array([1,2,3]))).shape == (3,). The reason for the difference may be in the implementation of the custom object's getbuffer...

Dov Grobgeld Over a year ago

I got it to work in the end. One issue that confused me is that you need to use the copy=False argument to np.array() in order to get a reference to the same memory.

Pierre de Buyl · Accepted Answer · 2014-08-17 08:20:53Z

0

You may also try np.asarray. I had success with it but I don't have access to hirsch so I can't test it.

BTW, Bakuriu has it right: you are creating a scalar array (ndim 0, size 1) of type np.object.

answered Aug 17, 2014 at 8:20

Pierre de Buyl

7,3232 gold badges18 silver badges23 bronze badges

Collectives™ on Stack Overflow

New style python buffer protocol and numpy arrays

2 Answers 2

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related