Python, Indexing and assigning to Np Array

Question

To improve the speed I would like to avoid forloops. I have a image array looking like : image = np.zeros_like(np.zeros(shape=(480,640,1)),dtype=np.uint8) and a typed np array Events with the following types dtype = [('x', '<f8'),('y', '<f8'),('grayVal','<u2') where 'x' = row and 'y' = column of the image array.

The Question is: How can I assign the grayVal in Events to all the x and y in the image ?

So far I tried (and more not displayable): The For Loop:

for event in Events:
    image[event['y'],event['x']] = event['grayVal']

and Indexing

events['y'].shape
(98210,)
events['x'].shape
(98210,)
events['grayVal'].shape
(98210,)
image[np.ix_(events['y'],events['x'])] = events['grayVal']

which somehow does not work due to the error message:

ValueError: shape mismatch: value array of shape (98210,) could not be broadcast to indexing result of shape (98210,98210,1)

What am I missing? Thanks for the help.

np.zeros(shape=(480,640), dtype=np.uint8) would describe an array that can be understood as a byte deep bitmap. I don't quite get where you would get the events from. Is it sparse, i.e. zero for most x,y pairs? Can you construct a minimal example (say for a 4 x4 matrix)? Leave out the ts, p,c for now unless you need them for the question. — roadrunner66
– roadrunner66, Commented Oct 20, 2020 at 8:18
Hey @roadrunner66, I added the for loop as example and deleted unuseful information - thanks. Yes, the Events array is Sparse and its zero for most pairs. — Daniel Klauser
– Daniel Klauser, Commented Oct 20, 2020 at 8:33
Have you experimented with a small example, say a size 5 Events? And compared indexing with [x,y] versus [ix_(x,y)]? — hpaulj
– hpaulj, Commented Oct 20, 2020 at 11:34
Hello @hpaulj, I was not able to produce a working example with the ix_ operator on my Events. I assumed that the ix_ assignment would be faster than the for loop. — Daniel Klauser
– Daniel Klauser, Commented Oct 20, 2020 at 12:52

hpaulj · Accepted Answer · 2020-10-20 18:42:44Z

Let's work with a small example, one we can actually examine and play with!

Make a structured array:

In [32]: dt = np.dtype([('x', int),('y', int) ,('grayVal','u2')])
In [33]: events = np.zeros(5, dt)
In [34]: events['x'] = np.arange(5)
In [35]: events['y'] = np.array([3,4,0,2,1])
In [36]: events['grayVal'] = np.arange(1,6)

To examine indexing lets make a nice 2d array:

In [38]: image = np.arange(25).reshape(5,5)
In [39]: image
Out[39]: 
array([[ 0,  1,  2,  3,  4],
       [ 5,  6,  7,  8,  9],
       [10, 11, 12, 13, 14],
       [15, 16, 17, 18, 19],
       [20, 21, 22, 23, 24]])

Look at what ix_ produces - 2 arrays that can broadcast against each other. A (5,1) and (1,5), which broadcast to (5,5):

In [40]: np.ix_(events['y'], events['x'])
Out[40]: 
(array([[3],
        [4],
        [0],
        [2],
        [1]]),
 array([[0, 1, 2, 3, 4]]))

Using those arrays to index image just shuffles values - the result is still a 2d array:

In [41]: image[np.ix_(events['y'], events['x'])]
Out[41]: 
array([[15, 16, 17, 18, 19],
       [20, 21, 22, 23, 24],
       [ 0,  1,  2,  3,  4],
       [10, 11, 12, 13, 14],
       [ 5,  6,  7,  8,  9]])

If instead we index with the arrays, not with the ix_ arrays:

In [42]: image[events['y'], events['x']]
Out[42]: array([15, 21,  2, 13,  9])

This is just the diagonal of the array produced with ix_. Indexing with a (n,) and (n,) arrays produces a (n,) array of values (as opposed to the ix_ (n,n) array).

So starting with a zeros image, we can assign values with:

In [43]: image= np.zeros((5,5), 'u2')
In [44]: image[events['y'], events['x']]=events['grayVal']
In [45]: image
Out[45]: 
array([[0, 0, 3, 0, 0],
       [0, 0, 0, 0, 5],
       [0, 0, 0, 4, 0],
       [1, 0, 0, 0, 0],
       [0, 2, 0, 0, 0]], dtype=uint16)

roadrunner66 · Accepted Answer · 2020-10-20 08:38:05Z

1

I can only think of a slow version, with a for loop for now. But that could be OK if the array is sparse. Maybe someone else can vectorize that.

import numpy as np

image =  np.zeros(shape=(3,4 ),dtype=np.uint8)  # image is empty
 
# evy is just a bag of nonzero pixels

evy=np.zeros(shape=(3), dtype = [('x', '<u2'),('y', '<u2') ,('grayVal','<u2') ])
evy[0]=(1,1,128)
evy[1]=(0,0,1)
evy[2] =(2,3,255) 
 #slow version
for i in range(3):
    image[evy[i][0],evy[i][1]]=evy[i][2]

output:

array([[  1,   0,   0,   0],
       [  0, 128,   0,   0],
       [  0,   0,   0, 255]], dtype=uint8)

answered Oct 20, 2020 at 8:38

roadrunner66

7,9914 gold badges34 silver badges39 bronze badges

2 Comments

hpaulj Over a year ago

Isn't this the same iteration as the OP's, for event in Events:? That is evy[i] is just a record of evy, It can be indexed as a tuple or a record, evy[i][0] or evy[i]['x'].

roadrunner66 Over a year ago

@hpaulj Yes, It is not a vectorized solution. I was just trying to help the OP (iteration did not exist in his first version, his version did not work at all) to formulate his question more clearly and to provide a working example for the slow (for loop style) iteration. I'm still trying to understand your proper, vector solution.

Collectives™ on Stack Overflow

Python, Indexing and assigning to Np Array

2 Answers 2

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related