26

I'm writing a preprocessor in python, part of which works with an AST.

There is a render() method that takes care of converting various statements to source code.

Now, I have it like this (shortened):

def render(self, s):
    """ Render a statement by type. """

    # code block (used in structures)
    if isinstance(s, S_Block):
        # delegate to private method that does the work
        return self._render_block(s)

    # empty statement
    if isinstance(s, S_Empty):
        return self._render_empty(s)

    # a function declaration
    if isinstance(s, S_Function):
        return self._render_function(s)

    # ...

As you can see, it's tedious, prone to errors and the code is quite long (I have many more kinds of statements).

The ideal solution would be (in Java syntax):

String render(S_Block s)
{
    // render block
}

String render(S_Empty s)
{
    // render empty statement
}

String render(S_Function s)
{
    // render function statement
}

// ...

Of course, python can't do this, because it has dynamic typing. When I searched for how to mimick method overloading, all answers just said "You don't want to do that in python". I guess that is true in some cases, but here kwargs is really not useful at all.

How would I do this in python, without the hideous kilometre-long sequence if type checking ifs, as shown above? Also, preferably a "pythonic" way to do so?

Note: There can be multiple "Renderer" implementations, which render the statements in different manners. I can't therefore move the rendering code to the statements and just call s.render(). It must be done in the renderer class.

(I've found some interesting "visitor" code, but I'm not sure if it's really the thing I want).

Improve this question
1
  • Check out the answers for this question. This one is technically a duplicate but it's received some different answers so I'm hesitant to flag it. Commented Aug 17, 2014 at 14:01

5 Answers 5

Reset to default
22

If you're using Python 3.4 (or are willing to install the backport for Python 2.6+), you can use functools.singledispatch for this*:

from functools import singledispatch

class S_Block(object): pass
class S_Empty(object): pass
class S_Function(object): pass


class Test(object):
    def __init__(self):
        self.render = singledispatch(self.render)
        self.render.register(S_Block, self._render_block)
        self.render.register(S_Empty, self._render_empty)
        self.render.register(S_Function, self._render_function)

    def render(self, s):
        raise TypeError("This type isn't supported: {}".format(type(s)))

    def _render_block(self, s):
        print("render block")

    def _render_empty(self, s):
        print("render empty")

    def _render_function(self, s):
        print("render function")


if __name__ == "__main__":
    t = Test()
    b = S_Block()
    f = S_Function()
    e = S_Empty()
    t.render(b)
    t.render(f)
    t.render(e)

Output:

render block
render function
render empty

*Code based on this gist.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Registering methods on an instance? Hmm, seems not ideal. Looking at the docs, I think you're supposed to use singledispatchmethod now. It was added in Python 3.8.
20

Would something like this work?

self.map = {
            S_Block : self._render_block,
            S_Empty : self._render_empty,
            S_Function: self._render_function
}
def render(self, s):
    return self.map[type(s)](s)

Keeping a reference to a class object as a key in a dictionary and having it's value be the function object you want to call will make your code shorter and less error prone. The only place an error could occur here would be in the definition of the dictionary. Or one of your internal functions of course.

Improve this answer

5 Comments

isinstance returns true for subclasses as well as the class itself. You can't match the semantics with a mapping.
@Joowani Great minds thing a like. The only thing you should consider is that in my implementation the dictionary is static and will be used across all instances of the class. Where your dictionary must be instantiated every time the function is used.
@roippi I never used isinstance? edit: Oh I see what you mean now. I hadn't considered that but it doesn't seem to be what the op is needing.
Interesting approach, and @roippi that's really not an issue here, I'm always matching for the "deepest child" of the hierarchy, eg. S_If, and not S_BranchingStatement (if I had one). I'll wait a while if something better turns up, but this certainly looks like a good technique (not perfect, but at least less verbose).
I've accepted this, because it works best for my projet and is easiest to understand. However, in general sense, the other answers may be better.
15

The overloading syntax you are looking for can be achieved using Guido van Rossum's multimethod decorator.

Here is a variant of the multimethod decorator which can decorate class methods (the original decorates plain functions). I've named the variant multidispatch to disambiguate it from the original:

import functools

def multidispatch(*types):
    def register(function):
        name = function.__name__
        mm = multidispatch.registry.get(name)
        if mm is None:
            @functools.wraps(function)
            def wrapper(self, *args):
                types = tuple(arg.__class__ for arg in args) 
                function = wrapper.typemap.get(types)
                if function is None:
                    raise TypeError("no match")
                return function(self, *args)
            wrapper.typemap = {}
            mm = multidispatch.registry[name] = wrapper
        if types in mm.typemap:
            raise TypeError("duplicate registration")
        mm.typemap[types] = function
        return mm
    return register
multidispatch.registry = {}

and it can be used like this:

class Foo(object):
    @multidispatch(str)
    def render(self, s):
        print('string: {}'.format(s))
    @multidispatch(float)
    def render(self, s):
        print('float: {}'.format(s))
    @multidispatch(float, int)
    def render(self, s, t):
        print('float, int: {}, {}'.format(s, t))

foo = Foo()
foo.render('text')
# string: text
foo.render(1.234)
# float: 1.234
foo.render(1.234, 2)
# float, int: 1.234, 2

The demo code above shows how to overload the Foo.render method based on the types of its arguments.

This code searches for exact matching types as opposed to checking for isinstance relationships. It could be modified to handle that (at the expense of making the lookups O(n) instead of O(1)) but since it sounds like you don't need this anyway, I'll leave the code in this simpler form.

Improve this answer

Comments

4

An alternate implementation with functools.singledispatch, using the decorators as defined in PEP-443:

from functools import singledispatch

class S_Unknown: pass
class S_Block: pass
class S_Empty: pass
class S_Function: pass
class S_SpecialBlock(S_Block): pass

@singledispatch
def render(s, **kwargs):
  print('Rendering an unknown type')

@render.register(S_Block)
def _(s, **kwargs):
  print('Rendering an S_Block')

@render.register(S_Empty)
def _(s, **kwargs):
  print('Rendering an S_Empty')

@render.register(S_Function)
def _(s, **kwargs):
  print('Rendering an S_Function')

if __name__ == '__main__':
  for t in [S_Unknown, S_Block, S_Empty, S_Function, S_SpecialBlock]:
    print(f'Passing an {t.__name__}')
    render(t())

This outputs

Passing an S_Unknown
Rendering an unknown type
Passing an S_Block
Rendering an S_Block
Passing an S_Empty
Rendering an S_Empty
Passing an S_Function
Rendering an S_Function
Passing an S_SpecialBlock
Rendering an S_Block

I like this version better than the one with the map because it has the same behavior as the implementation that uses isinstance(): when you pass an S_SpecialBlock, it passes it to the renderer that takes an S_Block.

Availability

As mentioned by dano in another answer, this works in Python 3.4+ and there is a backport for Python 2.6+.

If you have Python 3.7+, the register() attribute supports using type annotations:

@render.register
def _(s: S_Block, **kwargs):
  print('Rendering an S_Block')

Note

The one problem I can see is that you have to pass s as a positional argument, which means you can't do render(s=S_Block()).

Since single_dispatch uses the type of the first argument to figure out which version of render() to call, that would result in a TypeError - "render requires at least 1 positional argument" (cf source code)

Actually, I think it should be possible to use the keyword argument if there is only one... If you really need that then you can do something similar to this answer, which creates a custom decorator with a different wrapper. It would be a nice feature of Python as well.

Improve this answer

Comments

3

To add some performance measurements to the @unutbu 's answer:

@multimethod(float)
def foo(bar: float) -> str:
    return 'float: {}'.format(bar)

def foo_simple(bar):
    return 'string: {}'.format(bar)

import time

string_type = "test"
iterations = 10000000

start_time1 = time.time()
for i in range(iterations):
    foo(string_type)
end_time1 = time.time() - start_time1


start_time2 = time.time()
for i in range(iterations):
    foo_simple(string_type)
end_time2 = time.time() - start_time2

print("multimethod: " + str(end_time1))
print("standard: " + str(end_time2))

Returns:

> multimethod: 16.846999883651733
> standard:     4.509999990463257
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.