Python coroutines

Question

I have some experience with promises in Javascript. I'm pretty experienced with Python, but new to its coroutines, and there is a bit that I just don't understand: where does asynchrony float?

Consider the following minimal example:

async def gen(): await something return 42 

As I understand it, await something delays the execution of our function and allows the main program to run other bits. At some point something appeared a new result, and gen will have a result shortly after.

If gen and something are coroutines, then in all Internet wisdom they are generators. And the only way to find out when the generator has a new item available, afaik, is by polling it: x=gen(); next(x) x=gen(); next(x) . But it blocks! How does the scheduler “know” when x has a result? The answer cannot be “when something has a result”, because something must also be a generator (for it is a coroutine). And this argument is applied recursively.

I can’t get past this idea that at some point the process will just sit and wait synchronously.

Answer 1

The secret sauce here is the asyncio module . Your something object should be the most anticipated object and either depend on the more expected objects, or it should get on the Future object,

For example, asyncio.sleep() coroutine gives Future :

 @coroutine def sleep(delay, result=None, *, loop=None): """Coroutine that completes after a given time (in seconds).""" if delay == 0: yield return result if loop is None: loop = events.get_event_loop() future = loop.create_future() h = future._loop.call_later(delay, futures._set_result_unless_cancelled, future, result) try: return (yield from future) finally: h.cancel() 

(The syntax here uses the old generator syntax to stay backward compatible with older versions of Python 3).

Note that the future does not use await or yield from ; they simply use yield self until a condition is met . In the async.sleep() coroutine above, this condition is satisfied when the result was obtained (in the async.sleep() code above, via the function futures._set_result_unless_cancelled() , futures._set_result_unless_cancelled() after a delay).

Then the event loop continues to extract the next “result” from each expecting future that it controls (effectively polling them) until future signals are executed (by creating a StopIteration exception containing the results; return from the joint procedure would do this, eg). At this point, the coroutine that gave the future can be signaled to continue (either by sending a future result, or by StopIteration exception if the future raised anything other than StopIteration ).

So, for your example, the loop will start your gen() coroutine, and await something then (directly or indirectly) give the future. This future is polled until it raises the StopIteration value (signaling that it has been done) or raises another exception. If the future is done, coroutine.send(result) is executed, allowing it to jump to the return 42 line, raising a new StopIteration exception with this value, allowing the call to continue waiting on gen() , etc.