Running an event loop within its own thread

Question

I'm playing with Python's new(ish) asyncio stuff, trying to combine its event loop with traditional threading. I have written a class that runs the event loop in its own thread, to isolate it, and then provide a (synchronous) method that runs a coroutine on that loop and returns the result. (I realise this makes it a somewhat pointless example, because it necessarily serialises everything, but it's just as a proof-of-concept).

import asyncio
import aiohttp
from threading import Thread


class Fetcher(object):
    def __init__(self):
        self._loop = asyncio.new_event_loop()
        # FIXME Do I need this? It works either way...
        #asyncio.set_event_loop(self._loop)

        self._session = aiohttp.ClientSession(loop=self._loop)

        self._thread = Thread(target=self._loop.run_forever)
        self._thread.start()

    def __enter__(self):
        return self

    def __exit__(self, *e):
        self._session.close()
        self._loop.call_soon_threadsafe(self._loop.stop)
        self._thread.join()
        self._loop.close()

    def __call__(self, url:str) -> str:
        # FIXME Can I not get a future from some method of the loop?
        future = asyncio.run_coroutine_threadsafe(self._get_response(url), self._loop)
        return future.result()

    async def _get_response(self, url:str) -> str:
        async with self._session.get(url) as response:
            assert response.status == 200
            return await response.text()


if __name__ == "__main__":
    with Fetcher() as fetcher:
        while True:
            x = input("> ")

            if x.lower() == "exit":
                break

            try:
                print(fetcher(x))
            except Exception as e:
                print(f"WTF? {e.__class__.__name__}")

To avoid this sounding too much like a "Code Review" question, what is the purpose of asynchio.set_event_loop and do I need it in the above? It works fine with and without. Moreover, is there a loop-level method to invoke a coroutine and return a future? It seems a bit odd to do this with a module level function.

smart-developer · Accepted Answer · 2019-11-14 09:21:09Z

3

You would need to use set_event_loop if you called get_event_loop anywhere and wanted it to return the loop created when you called new_event_loop.

From the docs

If there’s need to set this loop as the event loop for the current context, set_event_loop() must be called explicitly.

Since you do not call get_event_loop anywhere in your example, you can omit the call to set_event_loop.

edited Nov 14, 2019 at 9:21

smart-developer

1971 gold badge4 silver badges14 bronze badges

answered Jul 11, 2017 at 16:15

dirn

21.1k7 gold badges75 silver badges76 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Xophmeister Over a year ago

What does "context" mean here? It can't mean "scope" as then there would be no need to call set_event_loop anywhere, unless for the odd case where you had multiple loops in the same scope. Is it a good idea to use set_event_loop regardless of the lack of get_event_loop, just in case the latter is called elsewhere in the codebase?

dirn Over a year ago

It will vary based on the policy used, but the default policy defines context as the current thread. And yes, if there's a chance the same loop will be used elsewhere, then you should call set_event_loop.

Xophmeister Over a year ago

If the default policy "manages an event loop per thread", does that mean I don't explicitly need to call asyncio.new_event_loop()? That is, does each thread have an implicit event loop, which will be returned by get_event_loop when it's called within the same context? (Obviously, in my code above, I create a new loop outside the new thread, but that can be changed easily enough.)

dirn Over a year ago

Correct. You can start with a call to get_event_loop. You only need to use new_event_loop when you don't want that loop or it's been closed.

Xophmeister Over a year ago

Awesome: Thanks :)

|

Matthijs · Accepted Answer · 2020-04-11 15:12:42Z

0

I might be misinterpreting, but i think the comment by @dirn in the marked answer is incorrect in stating that get_event_loop works from a thread. See the following example:

import asyncio
import threading

async def hello():
    print('started hello')
    await asyncio.sleep(5)
    print('finished hello')

def threaded_func():
    el = asyncio.get_event_loop()
    el.run_until_complete(hello())

thread = threading.Thread(target=threaded_func)
thread.start()

This produces the following error:

RuntimeError: There is no current event loop in thread 'Thread-1'.

It can be fixed by:

 - el = asyncio.get_event_loop()
 + el = asyncio.new_event_loop()

The documentation also specifies that this trick (creating an eventloop by calling get_event_loop) only works on the main thread:

If there is no current event loop set in the current OS thread, the OS thread is main, and set_event_loop() has not yet been called, asyncio will create a new event loop and set it as the current one.

Finally, the docs also recommend to use get_running_loop instead of get_event_loop if you're on version 3.7 or higher

answered Apr 11, 2020 at 15:12

Matthijs

4894 silver badges17 bronze badges

1 Comment

Quanta Over a year ago

I am pretty sure they meant to say- call set_event_loop() before calling get_event_loop() but yeah get_running_loop() sounds better.

Collectives™ on Stack Overflow

Running an event loop within its own thread

2 Answers 2

6 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related