72

I have the following code:

async some_callback(args):
    await some_function()

and I need to give it to a Thread as a target:

_thread = threading.Thread(target=some_callback, args=("some text"))
_thread.start()

The error that I get is "some_callback is never awaited".
Any ideas how can I solve this problem?

2
  • 2
    What scenarios necessitate passing an asyncio Task to a thread? Commented Sep 7, 2022 at 0:45
  • 1
    @thegreatcoder possibly to run long running async tasks in a web application where the initial response is expected to be swift. Commented Jun 22, 2024 at 16:11

7 Answers 7

87

As of Python 3.7, you can use asyncio.run() which is a bit more straightforward than loop.run_until_complete():

import asyncio

_thread = threading.Thread(target=asyncio.run, args=(some_callback("some text"),))
_thread.start()
Sign up to request clarification or add additional context in comments.

5 Comments

How does this actually launch something in a thread? It looked strange to me, and then I tested it and the some_callback does indeed execute before you start the thread, like it looks like it is going to. You know, because you are calling a sync function and all. No?
@anton Not sure to understand the question. some_callback("some_text") returns a coroutine object, see the doc: "simply calling a coroutine will not schedule it to be executed" and "To actually run a coroutine, asyncio provides three main mechanisms", among which asyncio.run. In my answer, asyncio.run is the target of the thread and takes the coroutine object returned by the some_callback function as argument, it is therefore executed in the thread
sure, not sure what my thinking was back then but 3 months ago I was still struggling to come to terms with async (even more than now anyway!) so probably was trying to pass a normal function. Anyway, sorry for the noise!
This might be problematic because the coroutine is created in a different event loop than the loop in which it is run. Something like this, while more verbose, is probably better.
83

You can do it by adding function between to execute async:

import asyncio

async def some_callback(args):
    await some_function()

def between_callback(args):
    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(loop)

    loop.run_until_complete(some_callback(args))
    loop.close()

_thread = threading.Thread(target=between_callback, args=("some text"))
_thread.start()

6 Comments

What scenarios necessitate passing an asyncio Task to a thread?
Won't the thread be blocked by python GIL?
Thread will be blocked if you run CPU-bound code. Thread will not be blocked with IO-bound code or sleep.
Ok, can you describe your usecase or scenario where you found it useful to run asyncio tasks on threads?
Example usage: create threads in asyncio can be only one way to run legacy code (when same function/library to run with asyncio not exists yet).
|
5

After Python 3.7, you can use asyncio.run()

import asyncio
import threading

async def some_callback(args):
    await some_function()

def wrap_async_func(args):
    asyncio.run(some_callback(args))

_thread = threading.Thread(target=wrap_async_func, args=("some text"))
_thread.start()

Comments

1

Didn't quite work for me, if you are trying to let it run next to your code (example: discord wait for emote reaction aswell as add emotes), you could try to use:

asyncio.get_event_loop().create_task(FUNKTION(ARGUMENT))

Comments

1

I recommend not mixing threads with asyncio, I wasted a day trying to replicate all examples here and still got the error "cannot schedule new futures after interpreter shutdown". It seems the asyncio loop is getting shut down by its thread at some point in my case, maybe while switching between threads, and the lib I am using has many async functions in its internals (that get called by the method I'm using).

I only got rid of this error by removing asyncio from threads. I think it's a valid warning for those who may experience a similar issue.

Comments

1

[mar-2024] this work for me.

import asyncio

async def __main(num1: int, num2: int):
  tasks = []
  tasks.append(print1(num1))
  tasks.append(print2(num2))
  print('starting main()')
  # Run the tasks concurrently
  await asyncio.gather(*tasks)

  # above could be simpler
  task1 = asyncio.create_task(print1)
  task2 = asyncio.create_task(print2)
  await task1
  await task2

And invoke the async function like this:

import threading

def run():
  # new thread
  thread = threading.Thread()
  thread.target = asyncio.run(__main(1, 2))
  thread.run() # or thread.start()

there's no need to pass args. I suppose within the synchronous context asyncio.run runs normally as synchronous. The Async behavioir is implicit in the line where the await ... declaration is specified.

6 Comments

This does not work. asyncio.run(__main(1, 2)) runs to completion, and then assigns the result to thread.target. There is no warning if the result of __main is None, but if it was an integer for example, you would get an error. See norbeq's answer for how to really do it stackoverflow.com/a/59645689/633921
You mean passing the target and args in Thread contructor, it works as well. But this way worked for me with no problems.
Open a Python repl type python or python3 whichever it is on your system, then type this into it: py import time import threading def blocking_sleep_test(): time.sleep(3) print("slept 3s") If you now call blocking_sleep_test(), it will block for 3s so you can't type, then it prints and unblock. This verifies how the blocking_sleep_test() method works.
Part 2: Now enter this: thread = threading.Thread(None, blocking_sleep_test). It executes immediately (no blocking) and if you wait for 3 seconds, nothing will printed, because the thread hasn't started yet. Then enter thread.start(). It will execute immeiately again (no blocking), and after 3 seconds it will print "slept 3s". That's how threading.Thread works, but more importantly that's how you verify that threading.Thread works the way you think it does.
Part 3: Now this is what you did: mjesbars_thread = threading.Thread() and mjesbars_thread.target = blocking_sleep_test() You will notice immediately after typing in the second line that it will block for 3s and print. That's because you are not assigning the method as the "target", you are running the method and assigning the return value (None). Then when you run mjesbars_thread.start() nothing will happen, because your thread has no target.
Part 4: Futhermore, there is no target property for threading.Thread(). So even if you would set the method instead of the return value as the "target" property, it would still do nothing when you run thread.start(). The fact that you also mix in an async method and asyncio.run() into this has absolutely no impact to your code, because that runs to completion before the thread even starts.
0

Just wrap your async function with this method and run the thread. _target is name of function in string and **args all your args. in addition to Norbeck solution

def middleware_callback(**args,_target):
   _proc = asyncio.new_event_loop()
   asyncio.set_event_loop(_proc)
   _proc.run_until_complete(getattr(sys.modules[__name__], _target)(**args)
   _proc.close()

1 Comment

this is a horrible implementation, it has errors. please do not use this

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.