Pythonic way to efficiently handle variable number of return args

Question

So I have a function that can either work quietly or verbosely. In quiet mode it produces an output. In verbose mode it also saves intermediate calculations to a list, though doing so takes extra computation in itself.

Before you ask, yes, this is an identified bottleneck for optimization, and the verbose output is rarely needed so that's fine.

So the question is, what's the most pythonic way to efficiently handle a function which may or may not return a second value? I suspect a pythonic way would be named tuples or dictionary output, e.g.

def f(x,verbose=False):
    result = 0
    verbosity = []
    for _ in x:
        foo = # something quick to calculate
        result += foo
        if verbose:
            verbosity += # something slow to calculate based on foo
    return {"result":result, "verbosity":verbosity}

But that requires constructing a dict when it's not needed.

Some alternatives are:

# "verbose" changes syntax of return value, yuck!
return result if verbose else (result,verbosity)

or using a mutable argument

def f(x,verbosity=None):
    if verbosity:
        assert verbosity==[[]]
    result = 0
    for _ in x:
        foo = # something quick to calculate
        result += foo
        if verbosity:
            # hard coded value, yuck
            verbosity[0] += # something slow to calculate based on foo
    return result

# for verbose results call as
verbosity = [[]]
f(x,verbosity)

Any better ideas?

Which version(s) of Python? If 3.x you could always return a list/tuple, and use result, *other_stuff = f(x, verbosity). See also e.g. stackoverflow.com/q/1893304/3001761, stackoverflow.com/q/14147675/3001761, stackoverflow.com/q/10223427/3001761, ... — jonrsharpe
– jonrsharpe, Commented Jul 28, 2015 at 14:46
...although I don't like the feel of that either, as non-verbose calls to f have to know about discarding other stuff. The mutable default argument avoids that issue. — Sideshow Bob
– Sideshow Bob, Commented Jul 28, 2015 at 14:53
The dictionary doesn't just provide a container for the two values; it provides a consistent interface for the caller of f, who doesn't have to know the value of the verbosity argument to use the return value. In other words, the existence (and not any particular value) of the verbosity argument makes the dict necessary. — chepner
– chepner, Commented Jul 28, 2015 at 14:53
The mutable argument version would be neater if you test by identity, is not None, then you can just pass [] instead of [[]]. — jonrsharpe
– jonrsharpe, Commented Jul 28, 2015 at 14:53

score 2 · Accepted Answer · 2015-07-28 15:41:02Z

2

Don't return verbosity. Make it an optional function argument, passed in by the caller, mutated in the function if not empty.

The non-pythonic part of some answers is the need to test the structure of the return value. Passing mutable arguments for optional processing avoids this ugliness.

edited Jul 28, 2015 at 15:41

answered Jul 28, 2015 at 15:02

user447688

Sign up to request clarification or add additional context in comments.

1 Comment

Sideshow Bob Over a year ago

Ultimately I think this makes most sense, so I'll accept it. I'm not convinced you have to test return value structure though, as I had imagined the verbose argument to hold a constant at compile time anyway, though I guess somebody could have a call that changes at runtime.

Phylogenesis · Accepted Answer · 2015-07-28 15:18:16Z

0

I like the first option, but instead of passing a verbose parameter in the function call, return a tuple of a quick result and a lazily-evaluated function:

import time

def getResult(x):
    quickResult = x * 2

    def verboseResult():
        time.sleep(5)
        return quickResult * 2

    return (quickResult, verboseResult)

# Returns immediately
(quickResult, verboseResult) = getResult(2)

print(quickResult)     # Prints immediately
print(verboseResult()) # Prints after running the long-running function

answered Jul 28, 2015 at 15:18

Phylogenesis

7,90024 silver badges27 bronze badges

2 Comments

Sideshow Bob Over a year ago

I haven't benchmarked this but won't the function definition for verboseResult() take significant time? Or if it doesn't then the binding of quickResult to its outer context will? Aside from that I agree this looks nicely pythonic.

Sideshow Bob Over a year ago

Also alas I don't see this working in my original example because verbosity needs access to a range of valued throughout the for loop. If we're saving these on the off chance verboseResult() gets called later then we're wasting time and memory (genuinely quite a lot in my case).

Collectives™ on Stack Overflow

Pythonic way to efficiently handle variable number of return args

2 Answers 2

1 Comment

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related