Make my own function as asyncio function in python

Question

I would like to use asyncio module in Python to achieve doing request tasks in parallel because my current request tasks works in sequence, which means it is blocking.

I have read the documents of asyncio module in Python, and I have wrote some simple code as follows, however it doesn't work as I thought.

import asyncio
class Demo(object):

    def demo(self):
        loop = asyncio.get_event_loop()
        tasks = [task1.verison(), task2.verison()]
        result = loop.run_until_complete(asyncio.wait(tasks))
        loop.close()
        print(result)

class Task():
   @asyncio.coroutine
   def version(self):
       print('before')
       result = yield from differenttask.GetVersion() 
       # result = yield from asyncio.sleep(1)
       print('after')

I found out that all the example they give use asyncio function to make the non-blocking works, how to make own function works as a asyncio?

What I want to achieve is that for a task it will execute the request and doesn't wait the response then it switch to next task. When I tried this: I get RuntimeError: Task got bad yield: 'hostname', which hostname is one item in my expected result.

so as @AndrewSvetlov said, differentask.GetVersion() is a regular synchronous function. I have tried the second method suggested in similar post, --- the one Keep your synchronous implementation of searching...blabla

@asyncio.coroutine
def version(self):
    return (yield from asyncio.get_event_loop().run_in_executor(None, self._proxy.GetVersion()))

And it still doesn't work, Now the error is Task exception was never retrieved future: <Task finished coro=<Task.version() done, defined at /root/syi.py:34> exception=TypeError("'dict' object is not callable",)>

I'm not sure if I understand if it right, please advice.

What I want to achieve is that for a task it will execute the request and doesn't wait the response then it switch to next task. I have error : RuntimeError: Task got bad yield: 'hostname' hostname is one item in my result. — ZhouQuan
– ZhouQuan, Commented Mar 8, 2016 at 13:33
differenttask.GetVersion() should be coroutine to be used with await syntax. Looks like you have a regular function. — Andrew Svetlov
– Andrew Svetlov, Commented Mar 9, 2016 at 11:24
@AndrewSvetlov Yes, GetVersion() is a regular synchronous function. I have tried the second way suggested in task got bad yield, which makes a regular function as async function if I understand it right. But it still doesn't work...I will add my changes in my post. — ZhouQuan
– ZhouQuan, Commented Mar 9, 2016 at 13:16

Andrew Svetlov · Accepted Answer · 2016-03-09 13:32:38Z

1

Change to

@asyncio.coroutine
def version(self):
    return (yield from asyncio.get_event_loop()
            .run_in_executor(None, self._proxy.GetVersion))

Please pay attention self._proxy.GetVersion is not called here but a reference to function is passed into the loop executor.

Now all IO performed by GetVersion() is still synchronous but executed in a thread pool.

It may have benefits for you or may not. If the whole program uses thread pool based solution only you need concurrent.futures.ThreadPool perhaps, not asyncio. If the most part of the application is built on top of asynchronous libraries but only relative small part uses thread pools -- that's fine.

answered Mar 9, 2016 at 13:32

Andrew Svetlov

17.5k8 gold badges70 silver badges72 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

ZhouQuan Over a year ago

I got this error TypeError: coroutines cannot be used with run_in_executor(), do you know why? And I have checked concurrent.futures.ThreadPool and as you suggested, I think it suits to my scenarios better than asyncio.

Andrew Svetlov Over a year ago

self._proxy.GetVersion should be a regular function for run_in_executor() usage. Your version() wrapper should be a coroutine to be used with yield from statement.

Collectives™ on Stack Overflow

Make my own function as asyncio function in python

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related