Why get_event_loop Cannot Be Used With run_in_executor

Question

Could anyone tell why the code below

import asyncio
import time
from concurrent.futures import ThreadPoolExecutor

ASYNC_INTERVAL = 0.1

def plain_hello_world(name):
    s = "Hello world "+str(name)
    print(s)
    return s

def plain_loop(name, t):
    start_time = time.time()
    prev_time = start_time
    while (time.time() - start_time < t):
        if time.time() - prev_time > ASYNC_INTERVAL:
            prev_time = time.time()
            plain_hello_world(name)

def task1():
    loop = asyncio.get_event_loop()
    task = loop.run_in_executor(None, plain_loop, "func", 1)
    loop.run_until_complete(task)

def task2():
    loop = asyncio.get_event_loop()
    task = loop.run_in_executor(None, task1)
    loop.run_until_complete(task)

if __name__ == "__main__":
    task2()

gets error below:

Traceback (most recent call last):
  File "asyncio_practice4.py", line 28, in <module>
    task2()
  File "asyncio_practice4.py", line 25, in task2
    loop.run_until_complete(task)
  File "/usr/lib/python3.6/asyncio/base_events.py", line 484, in run_until_complete
    return future.result()
  File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "asyncio_practice4.py", line 18, in task1
    loop = asyncio.get_event_loop()
  File "/usr/lib/python3.6/asyncio/events.py", line 694, in get_event_loop
    return get_event_loop_policy().get_event_loop()
  File "/usr/lib/python3.6/asyncio/events.py", line 602, in get_event_loop
    % threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'ThreadPoolExecutor-0_0'.

Question:

I do not understand why such error occurs.

Running task1() only is fine, but along with one another run_in_executor() it says no current evet loop. (But I think it should, I did not create a new thread)

Does anyone know what is going on?

And how to fix it assuming we can only work on task2()?

Note:

The reason to call 2 run_in_executor() is because code above is mimicking integrating a 3rd lib into asyncio code.

plain_hello_world(), plain_loop(), and task1() are codes in lib, which I cannot modify.

Assuming task1() runs for 100s and I do not want to wait for it, so I try to run it in executor as how other plain functions work with asyncio.

Edit: Based on the answer, here is the revision that works:

def task2():
    loop = asyncio.get_event_loop()
    def wrapper():
        asyncio.set_event_loop(asyncio.new_event_loop())
        task1()
    task = loop.run_in_executor(None, wrapper)
    loop.run_until_complete(task)

Though I am not sure how "correct" or good it is.

user4815162342 · Accepted Answer · 2020-03-16 19:06:19Z

1

I do not understand why such error occurs.

It occurs because task1 assumes that it will run in either the main thread (where get_event_loop() creates an event loop on-demand) or in a thread where set_event_loop was previously called to set up an event loop. Since run_in_executor invokes its function in a thread other than the main thread, and your code doesn't call set_event_loop() before invoking it, you get the error.

Assuming task1() runs for 100s and I do not want to wait for it, so I try to run it in executor as how other plain functions work with asyncio.

Running a sync function and not waiting for it is not something you do with asyncio, it's a job for regular threads. For example:

def task1_bg():
    def work():
        asyncio.set_event_loop(asyncio.new_event_loop())
        task1()
    threading.Thread(target=work).start()

if __name__ == '__main__':
    task1_bg()
    # do something else...

edited Mar 16, 2020 at 19:06

answered Mar 16, 2020 at 17:49

user4815162342

159k22 gold badges350 silver badges418 bronze badges

Sign up to request clarification or add additional context in comments.

10 Comments

adayoegi Over a year ago

I see, thanks. Is thread the only resort for sync functions? I thought run_in_executor() is the one to deal with those sync functions, and asyncio is a better substitute of thread (maybe I am wrong, though)

user4815162342 Over a year ago

@adayoegi run_in_executor is for an async code base. Asyncio is not a "substitute" for threads, it is a completely different approach. If your code base is already asyncio, you should be able to run the event loop, which task1 apparently doesn't let you do.

adayoegi Over a year ago

Thank you. I made a revision in edit. Do you think it makes sense?

user4815162342 Over a year ago

@adayoegi I am confused by your edit. That task2 implementation is clearly waiting for task1 to finish (because of run_until_complete). How is that while thing different than just calling task1() directly?

adayoegi Over a year ago

The result is same if we only have task1(), but is different if we have more than that. run_until_complete() is what I want here (it is a minimal example here, in real application it can be some functions calling task2(), while task2() itself is also async)

|

Collectives™ on Stack Overflow

Why get_event_loop Cannot Be Used With run_in_executor

1 Answer 1

10 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

10 Comments

Your Answer

Sign up or log in

Post as a guest

Related