Custom class with add to add with NumPy array

Question

I have a custom class implementing __add__ and __radd__ as

import numpy

class Foo(object):

    def __init__(self, val):
        self.val = val

    def __add__(self, other):
        print('__add__')
        print('type self = %s' % type(self))
        print('type other = %s' % type(other))
        return self.val + other

    def __radd__(self, other):
        print('__radd__')
        print('type self = %s' % type(self))
        print('type other = %s' % type(other))
        return other + self.val

I first test __add__

r1 = Foo(numpy.arange(3)) + numpy.arange(3,6)
print('type results = %s' % type(r1))
print('result = {}'.format(r1))

and it leads to the expected result

>>> __add__
>>> type self = <class '__main__.Foo'>
>>> type other = <type 'numpy.ndarray'>
>>> type results = <type 'numpy.ndarray'>
>>> result = [3  5  7]

However, testing __radd__

r2 = numpy.arange(3) + Foo(numpy.arange(3,6))
print('type results = %s' % type(r2))
print('result = {}'.format(r2))

I get

>>> __radd__
>>> type self = <class '__main__.Foo'>
>>> type other = <type 'int'>
>>> __radd__
>>> type self = <class '__main__.Foo'>
>>> type other = <type 'int'>
>>> __radd__
>>> type self = <class '__main__.Foo'>
>>> type other = <type 'int'>
>>> type results = <type 'numpy.ndarray'>
>>> result = [array([3, 4, 5]) array([4, 5, 6]) array([5, 6, 7])]

This doesn't make any sense to me. Does NumPy overload __add__ for arbitrary objects, which then takes priority over my __radd__? If yes, why would they do such a thing? Additionally, how can I avoid this, I really want to be able to add my custom class with a NumPy array to the left. Thanks.

Yeah, getting arithmetic operators to work nice with numpy arrays is a bit tricky, there is a lot of underlying machinery. I believe numpy provides mixins that let you do this relatively painlessly. I might be able to look into it later if no one else has the time. You can read more about it here — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Aug 29, 2017 at 20:03
Thank you for the link. I did not fully understand what these ufuncs are and how they work, but by setting __numpy_ufunc__ = None (__array_ufunc__ = None for NumPy 13.0+) in my class, I get the result that I wanted. — Max
– Max, Commented Aug 30, 2017 at 9:20
Also, you should post your answer and accept it. This was a pretty decent question. — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Aug 30, 2017 at 15:54

kabanus · Accepted Answer · 2020-03-10 07:34:25Z

This is hidden by the comments, but should be the answer.

By default, Numpy operations work on a per-element basis taking any arbitrary object, and then attempting to perform the operation by element (according to the broadcasting rules).

What this means, for example, is that given

class N:
    def __init__(self, x):
        self.x = x

    def __add__(self, other):
        return self.x + other

    def __radd__(self, other):
        return other + self.x

due to Pythons operator resolution

N(3) + np.array([1, 2, 3])

will get to the above __add__ with N(3) and the the entire array as other once, then perform a regular Numpy addition.

On the other hand

np.array([1, 2, 3]) + N(3)

will successfully enter Numpy's ufuncs (operator in this case) since they take arbitrary objects as the "other", and then attempt to perform successively:

1 + N(3)
2 + N(3)
3 + N(3)

This means that __add__ above will be called 3 times instead of once, once for each element, significantly slowing down operations. To disable this behavior, and make Numpy raise a NotImplementedError when taking an N object thus allowing the RHS overload radd to take over, add the following to the body of your class:

class N:
    ...
    __numpy_ufunc__ = None # Numpy up to 13.0
    __array_ufunc__ = None # Numpy 13.0 and above

If backward compatibility is not an issue only the second is needed.

Collectives™ on Stack Overflow

Custom class with add to add with NumPy array

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related