Asyncio and infinite loop

Question

@asyncio.coroutine
    def listener():
        while True:
            message = yield from websocket.recieve_message()
            if message:
                yield from handle(message)

loop = asyncio.get_event_loop()
loop.run_until_complete(listener())

Let's say i'm using websockets with asyncio. That means I recieve messages from websockets. And when I recieve a message, I want to handle the message but I'm loosing all the async thing with my code. Because the yield from handle(message) is definetly blocking... How could I find a way to make it non-blocking ? Like, handle multiple messages in the same time. Not having to wait the message to be handled before I can handle another message.

Thanks.

Usually you need a task per websocket for reading, writing to websocket may be performed asynchronously from other task. handle may be separate task also. Your code is not complete, so it's not easy to get what exactly do you need. — Andrew Svetlov
– Andrew Svetlov, Commented Dec 2, 2015 at 11:05
if this is the only coroutine you are calling, then the listener will block in the sense that it will run infinitely (because of the while true loop). If you have another coroutine running concurrently (with its own yield from statements) then asyncio will bounce back and forth between the yield from statements and it will thus cease to be 'blocking'. — songololo
– songololo, Commented Dec 2, 2015 at 11:51

Ethan Frey · Accepted Answer · 2016-02-02 13:54:16Z

If you don't care about the return value from handle message, you can simply create a new Task for it, which will run in the event loop alongside your websocket reader. Here is a simple example:

@asyncio.coroutine
def listener():
    while True:
        message = yield from websocket.recieve_message()
        if message:
            asyncio.ensure_future(handle(message))

ensure_future will create a task and attach it to the default event loop. Since the loop is already running, it will get processed alongside your websocket reader in parallel. In fact, if it is a slow-running I/O blocked task (like sending an email), you could easily have a few dozen handle(message) tasks running at once. They are created dynamically when needed, and destroyed when finished (with much lower overhead than spawning threads).

If you want a bit more control, you could simply write to an asyncio.Queue in the reader and have a task pool of a fixed size that can consume the queue, a typical pattern in multi-threaded or multi-process programming.

@asyncio.coroutine
def consumer(queue):
    while True:
        message = yield from queue.get()
        yield from handle(message)

@asyncio.coroutine
def listener(queue):
    for i in range(5):
         asyncio.ensure_future(consumer(queue))
    while True:
        message = yield from websocket.recieve_message()
        if message:
            yield from q.put(message)

q = asyncio.Queue()
loop = asyncio.get_event_loop()
loop.run_until_complete(listener(q))

Collectives™ on Stack Overflow

Asyncio and infinite loop

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related