Generating a remapped numpy array as a view.

Question

A toy-case for my problem:

I have a numpy array of size, say, 1000:

import numpy as np
a = np.arange(1000)

I also have a "projection array" p which is a mapping from a to another array b:

p = np.random.randint(0,1000,(1000,1000))

It is easy to get b from a using "fancy indexing":

b = a[p]

But b is not a view, as noted by several previous questions/answers and the numpy documentation.

Unfortunately, in my case only the values in a change over the course of a long simulation and using fancy indexing at each iteration to obtain b becomes very costly. I only read from b and do not modify it.

I understand it is not possible (yet) to solve this with fancy indexing.

I was wondering if anyone had a similar problem/bottleneck and came up with some other workaround?

Is all of b accessed in every iteration? If yes, there is no cheaper way than extracting all of b – you might need to improve the algorithm in other ways. If no, you don't need to build b, and instead of accessing b[i, j], you can use a[p[i, j]]. — Sven Marnach
– Sven Marnach, Commented Apr 4, 2012 at 21:19

Bi Rico · Accepted Answer · 2012-04-04 21:32:47Z

2

What your asking for isn't practical and that's why the numpy folks haven't implemented it. You could do it yourself with something like:

class FancyView(object):
    def __init__(self, array, index):
        self._array = array
        self._index = index.copy()
    def __array__(self):
        return self._array[self._index]
    def __getitem__(self, index):
        return self._array[self._index[index]]

b = FancyView(a, p)

But notice that the expensive a[p] operation will get called every time you use b as an array. There is no other practice way of making a 'view' of this kind. Numpy can get away with using views for basic slicing because it can manipulate the strides, but there is no way to do something like this using strides.

If you only need parts of b you might be able to get some time savings by indexing the fancy view instead of using it as an array.

edited Apr 4, 2012 at 21:32

answered Apr 4, 2012 at 21:27

Bi Rico

25.9k3 gold badges57 silver badges75 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

upperBound Over a year ago

That is what I was worried about :-) Thanks anyway.

Collectives™ on Stack Overflow

Generating a remapped numpy array as a view.

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related