Python Async Functionality

Question

I'm trying to figure out how the async functionality works in Python. I have watched countless videos but I guess I'm not 'getting it'. My code looks as follows:

def run_watchers():
    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(loop)
    loop.run_until_complete(watcher_helper())
    loop.close()

async def watcher_helper():
    watchers = Watcher.objects.all()

    for watcher in watchers:
        print("Running watcher : " + str(watcher.pk))
        await watcher_helper2(watcher)

async def watcher_helper2(watcher):
    for i in range(1,1000000):
        x = i * 1000 / 2000

What makes sense to me is to have three functions. One to start the loop, second to iterate through the different options to execute and third to do the work.

I am expecting the following output:

Running watcher : 1
Running watcher : 2
...
...

Calculation done
Calculation done
...
...

however I am getting:

Running watcher : 1
Calculation done
Running watcher : 2
Calculation done
...
...

which obviously shows the calculations are not done in parallel. Any idea what I am doing wrong?

Mixing Django models (Watcher) with asyncio code kills concurrency. — Andrew Svetlov
– Andrew Svetlov, Commented May 2, 2018 at 23:14

Mikhail Gerasimov · Accepted Answer · 2018-05-02 14:14:45Z

2

asyncio can be used only to speedup multiple network I/O related functions (send/receive data through internet). While you wait some data from network (which may take long time) you usually idle. Using asyncio allows you to use this idle time for another useful job: for example, to start another parallel network request.

asyncio can't somehow speedup CPU-related job (which is what watcher_helper2 do in your example). While you multiply some numbers there's simply no idle time which can be used to do something different and to achieve benefit through that.

Read also this answer for more detailed explanation.

answered May 2, 2018 at 14:14

Mikhail Gerasimov

40.2k20 gold badges126 silver badges172 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

ceds Over a year ago

Thanks, I understand what you are saying. I replaced the watcher_helper2 with the dummy code which does not make sense to use in async mode. For my case I found the solution, which is to use asyncio.wait(tasks)

Collectives™ on Stack Overflow

Python Async Functionality

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related