Async function call with Tornado Python

Question

I'm trying to make a simple async call, using gen.coroutine function of Tornado. This is my current code:

from tornado import gen
import tornado.ioloop
import tornado.web


class MainHandler(tornado.web.RequestHandler):

    @gen.coroutine
    def get(self):
        q = self.get_argument('query')
        print q
        response = yield self.process(q)
        self.write(response)

    @gen.coroutine
    def process(self, query):
        # just a long loop
        for i in range(int(query)*100):
            for j in range(i):
                a = 10*10*10*10*10*10
        return {'processed': True}


def make_app():
    return tornado.web.Application([
        (r"/search", MainHandler),
    ])

if __name__ == "__main__":
    app = make_app()
    port = 8888
    print "listening on port: ", port
    app.listen(port)
    tornado.ioloop.IOLoop.current().start()

However, it is not behaving in an async manner. What am I doing wrong in this?

notorious.no · Accepted Answer · 2017-05-26 15:57:56Z

4

Your function is blocking the event loop and no other tasks can be handled until either the process() function completes or it relinquishes control back to the event loop. For situations like this, you can use simply yield None (it used to be yield gen.moment) to take a break and let the event loop run other tasks then resume processing. Example:

@gen.coroutine
def process(self, query):
    for i in range(int(query)*100):
        for j in range(i):
            a = 10*10*10*10*10*10
            if j % 500 == 0:
                yield None    # yield the inner loop

        if i % 500 == 0:
            yield None    # yield outer loop

    return {'processed': True}

Hopefully this helps you achieve your desired level of concurrency.

References

http://www.tornadoweb.org/en/stable/gen.html#tornado.gen.moment

edited May 26, 2017 at 15:57

answered May 26, 2017 at 3:51

notorious.no

5,1573 gold badges22 silver badges34 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

A. Jesse Jiryu Davis · Accepted Answer · 2017-05-23 18:22:32Z

3

Your "process" method does only calculation, so it never provides Tornado's event loop an opportunity to work on other tasks while "process" is running. Tornado can interleave concurrent network operations, however, it cannot run Python code in parallel. To parallelize a function like your "process" method requires multiple Python subprocesses.

answered May 23, 2017 at 18:22

A. Jesse Jiryu Davis

24.1k5 gold badges60 silver badges72 bronze badges

4 Comments

n00b Over a year ago

But shouldn't it be able to receive another request while it is doing the processing in the 'process' function?

A. Jesse Jiryu Davis Over a year ago

No. I can describe it another way: if there's no "yield" or "await" in your "process" method, then Tornado's event loop never has an opportunity to interrupt the method. Tornado can only do concurrent network operations, in methods that include a "yield" or "await" expression.

n00b Over a year ago

If in place of return {'processed': True}, i make it yield {'processed': True}?

A. Jesse Jiryu Davis Over a year ago

Still no. =) Tornado can only do concurrent network operations. It cannot run Python code in parallel. To parallelize a function like your "process" method requires multiple Python subprocesses.

Collectives™ on Stack Overflow

Async function call with Tornado Python

2 Answers 2

References

Comments

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

References

Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related