I have two very close questions. One practical, related to the real problem that I have, and one more theoretical. Let me start with the last:
Q1: Do asynchronous / standby + generators make sense?
In general, it is not too important to have an exit inside callbacks or promises. The problem (among other things) is that there may be races between several outputs, it is impossible to say when the generator ends, etc.
function* this_is_nonsense() { setTimeout(() => { yield 'a'; }, 500); setTimeout(() => { yield 'b'; }, 500); return 'c'; }
When using async / await these problems seem to mostly go away
function timeout(ns) { return new Promise((resolve) => setTimeout(resolve, ns)); } async function* this_could_maybe_make_sense() { // now stuff is sequenced instead of racing await timeout(500); yield 'a'; await timeout(500); yield 'b'; return 'c'; }
I assume that nothing like this is possible at the moment (please correct me if you are mistaken!). So my first question is: is there a fundamental problem with using generators with async / await? If not, is it on any roadmap? It seems that implementation will require more than just a merge of two functions.
Q2: How to write a lazy wiki scanner
This is a far-fetched version of the real code problem that I have.
Imagine that you are creating a function that bypasses Wikipedia, starting from any arbitrary page and repeating the following links (it doesn’t matter how - maybe depth, width or something else first). Suppose we can asynchronously scan Wikipedia pages to get related pages. Sort of:
async function traverseWikipedia(startLink) { const visited = {}; const result = []; function async visit(link) { if (visited[link]) { return; } visited[link] = true; result.push(link); const neighboringLinks = await getNeighboringWikipediaLinks(link); for (const i = 0; i < neighboringLinks.length; i++) { await visit(neighboringLinks[i]); }
This is great ... except that sometimes I only need the first 10 links and it will crawl through a lot of links. Sometimes I search for the first 10 that contain the substring "Francis". I need a lazy version of this.
Conceptually, instead of the return type Promise<Array<T>> I want Generator<T> or at least Generator<Promise<T>> . And I want the consumer to be able to execute arbitrary logic, including an asynchronous stop condition. Here is a fake, completely wrong (I guess) version:
function* async traverseWikipedia(startLink) { const visited = {}; function* async visit(link) { if (visited[link]) { return; } visited[link] = true; yield link; const neighboringLinks = await getNeighboringWikipediaLinks(link); for (const i = 0; i < neighboringLinks.length; i++) { yield* await visit(neighboringLinks[i]); } ); yield* await visit(startLink); }
This is the same nonsense from Q1 that combines async / wait + + generators, as if she were doing what I want. But if I had it, I could do this:
function find10LinksWithFrancis() { const results = [] for (const link of traverseWikipedia()) { if (link.indexOf('Francis') > -1) { results.push(link); } if (results.length === 10) { break; } } return results; }
But I could also look for a different number of results and rows, have asynchronous stop conditions, etc. For example, it can display 10 results, and when the user clicks a button, it continues scanning and shows the next 10.
I'm fine using promises / callbacks, without async / await, while I still get the same API for the consumer.
The calling generator can perform asynchronous actions. Therefore, I consider it possible to realize this when the generator function makes a promise to the neighbors, and gen.next with the resulting actual neighbors is called before the caller. Sort of:
function* traverseWikipedia(startLink) { const visited = {}; function* visit(link) { if (visited[link]) { return; } visited[link] = true; yield { value: link }; const neighboringLinks = yield { fetch: getNeighboringWikipediaLinks(link) }; for (const i = 0; i < neighboringLinks.length; i++) { yield* visit(neighboringLinks[i]); } ); yield* visit(startLink); }
But this is not very pleasant to use ... you need to determine the case of the actual result compared to the generator with a request to give him an answer, it is less typical, etc. I just need something similar to Generator<Promise<T>> and a relatively simple way to use it. Is it possible?
Edited to add: it occurred to me that you could implement this if the body of the generator function could access the generator that it somehow returned ... not sure how to do it, though
Edited again to add a consumer example, and note: I would be fine if the consumer were a generator, if that helps. I think this is possible with some kind of engine layer beneath it, reminiscent of sound reductions. Not sure if someone wrote such a thing.