3

This is a question about programming style: what is the most "Pythonic" way to parameterize a function (is that even the right word for it?)

Say I have a function (e.g. an ODE solver) that accepts as an argument another function of two arguments (e.g. the ODE itself).

def solver(fun):
   # Implementation goes here
   # fun is a function handle that accepts two args e.g. fun(t,y)

However, the function I would like to pass into solver is parameterized by a third value

def myFun(t,y,A):
   # Implementation goes here

I have been dealing with this situation using lambda functions as follows:

A = 4
solution = solver(lambda t,y:myFun(t,y,A))

I've recently been seeing some posts online telling me to avoid lambdas like the plague, and that Guido himself regrets allowing that feature. If lambdas are indeed terrible, what's the better "Pythonic" way to implement the above? Without lambda I run into the problem of being unable to access the global namespace, i.e. I want to do this:

A = 4
def myFunNotLambda(t,y):
    return myFun(t,y,A)
solution = solver(myFunNotLambda)

but the only way to do this seems to be making A global, which is definitely far worse than using lambda

6
  • 2
    there's also functools.partial Commented Jun 12, 2017 at 18:47
  • 3
    There's no need to avoid lambdas like the plague. That said, def can do anything lambda can; you just forgot to return the value. functools.partial also works, although it can't fix positional arguments that don't occur at the front. Commented Jun 12, 2017 at 18:49
  • Guido's main objection to lambda is when people use it to wrap a simple expression to pass to map (or filter) instead of directly executing the expression in a list comp or gen exp, eg map(lambda x:5*x+1, seq) vs (5*x+1 for x in seq). Notice that the mapversion incurs the overhead of a Python function call on every item in seq. Of course, this is irrelevant if the callable you pass to map is implemented in C, eg map(int, list_of_strings) is perfectly fine. Commented Jun 12, 2017 at 19:38
  • @user2357112 barring the missing return (fixed now), I think that still won't work since the value of A does not exist inside the myFunNotLambda namespace Commented Jun 12, 2017 at 20:32
  • 1
    @dkv: Why do you think it doesn't? Did you define myFunNotLambda outside the scope of A? Commented Jun 12, 2017 at 22:34

3 Answers 3

5

You can use functools.partial for that, like:

from functools import partial

A = 4
solution = solver(partial(myFun,A=A))

partial(..) constructs given a function (here myFunc) another function where the A parameter has now a default value with A.

Sign up to request clarification or add additional context in comments.

6 Comments

+1 This nicely decouples any logic of passing arguments to the caller's function. Additionally, this increases the flexibility of solver, effectively allowing for myFun to accept any number of constant arguments.
Is the overhead incurred from constructing this function the same as the overhead from constructing a lambda?
@dkv: constructing or calling? Because usually constructing happens only once and is thus not significant. This answer suggests that partial is faster than lambda.
@dkv partial returns a full fledged callable (looks like an instance of a class). It would only get constructed by the caller when they pass it in. If you have a loop with a lot of iterations, it's calling time you want to worry about. That said, I'd argue you're talking about a micro-optimization. I'd profile it and see if it's a problem first.
@dkv The construction of the partial object is a one-off cost, once it's constructed it doesn't need to be be construced again, unless you need a new one with a different A parameter. But when you call the partial object, it then has to call the original myFun, so you get two levels of Python function / method call, just like in your lambda-based solution.
|
3

Advice to "avoid lambda like the plague" is a serious exaggeration at best, and pure FUD at worst - grepping lambda in Python's standard library reveals literally thousands of matches.

While it is true that Guido has expressed later concern about lambda-dominated coding style, this was in context of (over)using functional constructs like map and reduce, which are better expressed in Python using list comprehensions and generator expressions.

When it comes to creating temporary functions like the one you need, however, there is exactly nothing wrong with using lambda; on the contrary, it's the best available tool for the job. If you still want to avoid lambda keyword, you can use a nested def:

def my_particular_fun(t, y):
    return my_fun(t, y, 4)

solution = solver(my_particular_fun)

As presented in other answers, it is also possible to use functools.partial to emulate a lambda, although it comes at a cost of fixing the name of the third parameter to my_fun.

10 Comments

The problem with my_particular_fun is that it incurs the cost of a double function call, and Python calls are relatively slow.
@PM2Ring That is the case with any solution other than rewriting my_fun to hard-code the A argument to 4. Also, worrying about the speed of a single additional function call without having any idea what the function is doing sounds like premature optimization.
You don't need to hard-code the A to a fixed value, you can make it available via a closure, as I show in my answer. OTOH, I normally just use a lambda in this situation, or occasionally partial, unless the solver is taking a long time & I'm desperate for speed-ups. :)
Fair enough, although the OP didn't mention they were using Numpy, but if they aren't they're probably using something else that can do fast mathematics, eg mpmath, which uses gmpy if it's available. And you are certainly correct that using a constant in Python avoids a name lookup, although we don't know that the OP's A actually is a constant. ;)
@dkv Unless you can rewrite my_fun, there is simply no avoiding the double call. lambda does it, the nested def does it, and functools.partial does it too. The additional function call will be noticable only if the rest of the function is truly trivial.
|
1

A quite efficient way to do this is to use functools.partial, but as has been pointed out, partial only lets you "freeze" final args. If you need something else, you can easily implement your own version using a closure.

This approach is actually a little more efficient than using a partial object, since when you call the partial it still has to call the original function that you pass in, so each call of the partial results in two calls, and Python function / method calls are relatively slow. If you're curious, take a look at the Python source code for functools.partial.

In this example, I've made A the second (pseudo) parameter of the function, since partial nicely handles the case where it's the last arg.

def my_partial(param):
    A = param
    print('Creating two_arg_func with A ==', A)
    def two_arg_func(t, y):
        # Do something with all the args
        return 'args', t, A, y
    return two_arg_func

def test(f):
    for u in range(10, 50, 10):
        print(f(u, u + 5))

test(my_partial(7))

output

Creating two_arg_func with A == 7
('args', 10, 7, 15)
('args', 20, 7, 25)
('args', 30, 7, 35)
('args', 40, 7, 45)

We don't really need param in my_partial, we can just use the passed in arg, since that's local to my_partial:

def my_partial(A):
    print('Creating two_arg_func with A ==', A)
    def two_arg_func(t, y):
        return 'args', t, A, y
    return two_arg_func

From your comments, I now understand that you want to be able to vary A. Of course you can do that by calling partial or my_partial again, but if you want to modify A a lot that's not so efficient.

Your comments indicate that you want to modify A in the global context, so you might as well just use a global. You don't need to actually put the code modifying A into the global context, you can wrap it in a function, but of course you will need to use the global directive in the function that modifies A. You do not need the global directive in any function which merely reads the value of A, though.

Here's a short demo.

def two_arg_func(t, y):
    # Do something with the args and with A
    return 'args', t, A, y

def solve(f):
    for u in range(10, 40, 10):
        print('SOLVER', f(u, u + 5))

def test(f):
    global A
    for A in range(7, 10):
        print(A)
        solve(f)

test(two_arg_func)

output

7
SOLVER ('args', 10, 7, 15)
SOLVER ('args', 20, 7, 25)
SOLVER ('args', 30, 7, 35)
8
SOLVER ('args', 10, 8, 15)
SOLVER ('args', 20, 8, 25)
SOLVER ('args', 30, 8, 35)
9
SOLVER ('args', 10, 9, 15)
SOLVER ('args', 20, 9, 25)
SOLVER ('args', 30, 9, 35)

However, that previous solution is somewhat unsatisfactory, since the main thrust of your question was how to do this without using a global. So here's a slight variation on that code. Instead of putting A in the global namespace, we attach it to two_arg_func as a function attribute. We can do this because a Python function is an object, and it already has a whole bunch of attributes; two you may be familiar with are __name__ and __doc__. Anyway, here's the new code, it prints the same output as the previous version.

def two_arg_func(t, y):
    A = two_arg_func.A
    # Do something with the args and with A
    return 'args', t, A, y

def solve(f):
    for u in range(10, 40, 10):
        print('SOLVER', f(u, u + 5))

def test(f):
    for A in range(7, 10):
        print(A)
        f.A = A
        solve(f)

test(two_arg_func)

7 Comments

Doesn't my_partial have the same issue of a double call? i.e. when you call test, it calls my_partial four times, and each of those four times it defines and immediately calls two_arg_func?
@dkv No, in my code my_partial is only called once, in the final line. two_arg_func is only defined once, just after Creating two_arg_func with A == 7 gets printed (in fact, I added that print call there to demonstrate that). It will only create a new function if you call my_partial again, eg if you need a different A value. The loop inside test makes 4 calls, and those calls are directly to the function that was returned by the one and only call to my_partial(7).
Oh, right, sorry, the print statement does make it very clear. Is that some built-in Python magic, or is there something obvious I'm missing?
@dkv I'm not quite sure what magic you're talking about. :) Python functions are first-class objects, so it's quite simple for them to be passed as arguments, or to be returned as return values. I guess one magic thing is how two_arg_func gets access to A. It does that via a closure cell. See this answer about closures which I also linked in the answer.
@dkv No, when you unroll the loop f is called 4 times, and that f is the function object returned by the single call to my_partial. However, from reading your comments to user4815162342 it appears that we've misunderstood what you want to do with A. We all thought that you wanted a fixed value of A but you don't. Oops! I'll update my answer shortly.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.