Can't Use 'await' in Fiber

[DavidRusso]

index_js.txt

See attached example. After the 'await' call, fibers.current is not defined. This makes it impossible to use 'await' in a Fiber that also does asynchronous processing with callbacks.

[benjamn]

This behavior is expected, but it's worth noting that a better system would have the following properties:

• Every async function acquires a new Fiber in which to do its work.
• That Fiber is yielded before each await and restored when the async function resumes execution.
• A new Promise.prototype.await (or perhaps Fiber.await) method can be used to await any promise "synchronously" (i.e. it just yields whatever Fiber is currently active until the promise resolves). An exception is thrown at runtime if Fiber.current is not defined.

This is how fibers work with async/await in Meteor, and it's such a powerful abstraction that we almost never need to use the Fiber or Future APIs directly anymore. It feels just like async/await, except that await expressions can appear not just in the body of async functions, but anywhere in the dynamic call stack of an async function (or an explicit Fiber).

@laverdet For what it's worth, Meteor could abandon a lot of our custom transformation and Promise.prototype.then-wrapping logic if the fibers library automatically integrated with async functions in this way. Would you be interested in a PR to implement that behavior?

[laverdet]

The Future library provides adapters to convert between JS-style promises and Fiber-style futures. An example of this:

const Future = require('fibers/future');
  
let task = Future.task(function() {
  let promise = Future.fromPromise(asyncStuff());
  let future = fiberStuff();
  Future.wait([ promise, future ]);
  console.log(promise.get(), future.get());
});

(async function() {
  await task.promise();
  console.log('done with both');
})();

function asyncStuff() {
  return new Promise(function(resolve, reject) {
    setTimeout(() => resolve('done with async'), 2000);
  });
}

function fiberStuff() {
  let future = new Future;
  setTimeout(() => future.return('done with fiber'), 2000);
  return future;
}

@benjamn it seems like such a modification would have surprising performance characteristics. Fibers are "lightweight" but not nearly as lightweight as the single-frame generators builtin to modern JS. The way promises and async functions are used tend to create several promises for a single RPC request. The basic usage example of fetch on MDN creates at least 4 intermediary promises.. for those to be fibers you'd see a lot of memory usage, don't you think?

[tcf909]

I'm running in to similar issue where I have some code that is trying to use async / await, but after an await the code calls other code that is trying to use a Fiber (which it is no longer running in).

Because in the stack the person uses an "async function(){}" we basically lose the ability for the rest of the code to run in a fiber.

Bringing Davids code in to the thread:

const fibers = require("fibers");

fibers(function() {
  
  (async function() {
    
    console.log(fibers.current);
    console.log(await asyncStuff());
    console.log(fibers.current); //fibers.current is no longer available
    console.log(asyncStuff2());

  })();  
  
}).run();

function async asyncStuff() {
   //Uses await
}

function asyncStuff2() {
  //Uses fiber
}

Since async does some magic around Promise we have no way to instrument the function so that on return to the main call stack in continues executing in to a fiber.

I think being able to mix Promises with Fibers is potentially very powerful as it allows people to use newer semantics where it makes sense and allows a complex codebase to not be entirely promise based.

Any suggestions on how to make sure async functions return execution in to a fiber after an await?

[tcf909]

@benjamn
I just want to make sure I understood...When you talk about what Meteor does today, you mean you do that through function calls vs some magic behind the scenes correct?

I.E.:

//This is what I think you mean:
async(function(){
  const result = await(async(() => {
    //I'm still running in a fiber
  }))
})

Vs:

async function(){
  const result = await async () => {
    //I wish I was running in a fiber, but alas I am not...
  }
}

Correct?

[carlo-quinonez]

Any suggestions on how to work around using promise-based libraries in a Fiber?

[tcf909]

You can still use promises, just not await.

The Fibers library has an example for converting promises to a fiber yield. It's pretty straight forward in that it is just passing the resume callback to the Promise then and catch functions

[jedwards1211]

If Meteor had only just now been created, they would have never messed with fibers, they would have just used async/await so that there's less API mismatch between server and client and none of this complexity of native v8 add-ons and promise wrapping.

[laverdet]

@jedwards1211 yeah duh :). I made fibers in 2011 because it was painfully obvious that the tools we had were no good. We didn't get generators in nodejs until September 2015 and async/await until February 2017.

[jedwards1211]

Yeah. What I'm actually annoyed about is that the author of Webdriverio kept a fiber-oriented approach in his most recent API breaking change, and it actually got more ungainly to use with async/await as a result. I landed here when I was searching for whether it would be possible to call a function that uses fibers from an async function. Maybe it actually is? But I think I remember reading for some reason it doesn't work with wdio-sync.