Futures Eliminate Time

Last updated: Saturday, April 20 2013

I was thinking a bit yesterday on Domenic's blog post "You're Missing the Point of Promises", and have some random thoughts about the real effect of Futures on code.

In short: Futures abstract away the concept of time from your code.

Our programming languages today simply aren't well-designed to handle time, where an operation can take a non-trivial amount of time to complete. If you don't want to just block until it finishes (effectively ignoring the time component), you have to go through all kinds of contortions to handle it properly.

In a good language like JS, the contortions aren't too terrible. You end up with callback hell, but if you've ever tried to do the same in languages like Java, you quickly realize how good we have it in JS.

Fundamentally, though, these contortions are just hacks. They're broken - the language doesn't actually cater toward them, and you lose a lot of functionality. One of the most important things you lose is the ability to respond to errors.

Functions Abstract over Space

Let's step back a moment, and consider this issue more abstractly.

What are functions? What do they, fundamentally, accomplish?

It's pretty simple (at least, if you're already in my head): functions abstract over space, allowing you to ignore where code is written in your source file. You can write code anywhere you want, and then "call" it, making it act as if it was written right where you want it. (It also lets you call that code multiple times, but that's a step beyond what I'm trying to do here, so I'll ignore it.)

Imagine, though, that you've been working in a language without any functions. This language has a lot of other modern features, like the ability to throw and catch errors, but it only recently added this newfangled "function" concept, and not everything is well-integrated yet. (Let's call it "Mauvascript".)

In particular, throwing and catching errors is based on source location. Writing code this is okay:

try {
  if( x == 0 )
    throw Error("Tried to divide by zero.");
  y = 1/x;
} catch(e) { ... }

This works exactly as expected - if x is zero, the error gets thrown, and caught by the try/catch block.

However, because functions are brand new, the following code doesn't work as expected:

function invert(x) {
  if( x == 0 )
    throw Error("Tried to divide by zero.");
  return 1/x;
}

try {
  y = invert(x);
} catch(e) { ... }

In a modern language, this transformation, moving the code around in the source file, doesn't have any confusing effects. The error will get thrown, propagate up the call chain, and be caught by the try/catch at the calling site.

But this isn't a modern language, it's Mauvascript. Functions are newfangled, and things like "errors should propagate up the call chain" are complicated and non-obvious. Instead, Mauvascript makes errors propagate up the lexical scope (that is, just looking at wider scopes in the source code) and makes uncaught errors leaving functions cause the function to return nothing.

You should be able to imagine the pain that people writing Mauvascript would have when using functions. If you use functions to start moving code around in space, suddenly errors stop working correctly! Dealing with any sort of error-handling becomes far more difficult.

You could imagine some conventions arising to deal with it, of course. A popular web server using Mauvascript, perhaps called Mode.ms, might develop the convention that every function has to return a 2-element array, with the first element holding the error object, if there was any, and the second holding the normal return value, if it completed successfully. Every function would need to wrap its entire contents in a try/catch, then, with the error return done in the catch. This is less convenient, but it's predictable, and that would allow a number of control-flow libraries to pop up to help make error-handling more convenient.

Fundamentally, though, all of this would just be papering over the base problem, which is that errors shouldn't be tied to the location of the code in the source file. Moving code around shouldn't have any effect on its behavior, which means that errors need to pay attention to the call stack, and propagate up to the calling site when uncaught within a function.

Futures Abstract over Time

Today's Javascript is in the exact same situation as our imaginary Mauvascript, except the new concept being dealt with is code moving around in time, rather than space. Long-running functions take callback arguments, which has the effect of shifting the block of code into the future to be called later (just like how functions shifted a block of code into another place in the source file).

The problems Mauvascript had with errors in functions are exactly like the problems we have today with errors in callbacks, because the language doesn't support async code natively.

Futures are the abstraction that fixes this. They let you schedule code to be run in the future, and if the code throws an error, you can catch it later. Nothing special needs to be done on your side, no special conventions need to be adhered to. Whenever you chain code to run after a future, you can set two callbacks, one for normal code and one for the error case. If you omit the error code, it'll just automatically skip your normal code and jump to the next code in the chain until it finds something that can handle it.

This is exactly equivalent to the behavior of proper functions and errors - the error bubbles up to the call site, and if you don't catch it, continues bubbling up, skipping over any code you write between the erroring call and the end of your function.

This is the true value of futures - not that they make it easier to write async code (callbacks and convention do that reasonably well already), but that they capture the notion of error flow properly across async boundaries.

An Even Better Future?

It seems likely to me that, now that Futures are finally gaining proper popularity (both in JS and in more traditional languages like C# and C++), some language will eventually pop up with more integrated syntax for async code. I'm not sure what it would look like, though. Does anyone know any examples of experimental languages that do something like this?

C# already has syntactic support for this, in version 5, using the async and await keywords: http://msdn.microsoft.com/en-gb/library/vstudio/hh191443.aspx

I believe the compiler is clever enough to make error handling work as you would expect: E.g. you wrap async calls in try/catches and any exceptions thrown by the async code get handled in the wrapping catch.

Reply?

In terms of languages with a more integrated syntax I was thinking one should be created in which everything is a promise. Then you'd be able to just write your code entirely as if it was synchronous, as values became available they would cause code that was now able to be evaluated to get evaluated. It feels like the concept would be somehow analogous to Haskell's laziness.

I started writing a transpiler for something like this that takes a subset of JavaScript and converts it all to promise based code. Converting loops and dictionaries is interesting stuff. Code is here: https://github.com/cjwainwright/promise-js

Would love to know if there's already something out there like this, or if anyone else has taken this idea further? Could even do the same with streams I expect.

Reply?

Yeah, async/await are definitely a start. The automatic transformation they do is really interesting.

It sounds like this code:

function foo() {
  bar = 1;
  baz = await qux();
  return bar+baz;
}

is equivalent to this promise-based code:

function foo() {
  bar = 1;
  return qux().then(baz=>bar+baz);

A try/catch is more complex, but I think it would do this:

function() {
  a = 1;
  try {
    b = 2;
    c = await d();
  } catch(e) {
    c = 3;
  }
  return a+b+c;
}

becomes

function() {
  a = 1;
  try {
    b = 2;
    temp = d();
  } catch(e) {
    c = 3;
    return a+b+c;
  }
  return temp.then(
    c=>a+b+c,
    e=>{c = 3; return a+b+c;});
}

This requires some data-flow tracking to work around things, and some duplication, but I think it's the correct behavior.

This also assumes that it's okay for functions to sometimes return promises and sometimes not. If that's not okay, and the mere presence of await in any code-path turns it into a promise (or, like Chris in comment #2 mentions, the language inherently does everything async), then the refactoring is a bit simpler.

Multiple awaits would produce nested promises, I believe. I'm not going to try and unfold that in an example. ^_^

Reply?

Yeah, that's probably what a language that natively addressed async would do. If everything was always async, then you could even do synchronous promise processing sometimes, when a called function doesn't rely on anything async. That would be pretty cool.

Reply?

I'll also note that my colleagues on Chrome looked into adding async/await to JS. This was pre-Future, though, so we had to do some funky stuff to deal with calling sites that didn't realize the function they were calling was async. Doing CPS transforms on arbitrary JS is, um, fun.

If you have a typed language, this is easier - it can be a compiler error to call an async function without await.

Now that we have Futures in JS, we can do another route, and just have async functions return a Future. It'd be like generators - some special function form that allowed the use of a special keyword in their body (for generators, it's "function* foo(){}" and "yield"). The return value would be a Future.

In order to use that Future "natively", you'd have to declare your function async as well, but otherwise you could just take the Future and operate on it normally.

Reply?

As Jon mentioned, C# already has async / await support built into the language, which is available in a more generalized form in F# as computation expressions or workflows, which (this shouldn't be a surprise) are actually sugar for F#'s monadic / functor-based datatype polymorphism. Declarative concurrency is common in functional programming, and your description is an good summary of how declarative concurrency abstracts over time. The nice thing about using a monadic approach is that you can also take advantage of the other applicatives that propagate state; the ugly thing is that combining or stacking monadic strategies gets painful in a hurry in statically-typed functional languages.

That said, this discussion shows that care is required when trying to abstract promises out into a "pure" monad -- Promises/A+ takes a pragmatic approach based on the need to abstract over time while also simplifying error-handling, and this is a good property to preserve.

Back in the imperative domain, the fork / join syntax of coroutines, which goes back to Tony Hoare's communicating sequential processes (CSP), if not further, were another way to implement concurrency in single-threaded runtimes.

Reply?

Also, back in the JS realm, there's IcedCoffeeScript and its antecedent TameJS, which are a little like Promise.js in the way they extend the syntax of the language and rely on transpilation to handle the CPS transforms. The real issue of using something like that is twofold: 1. how do you use something like that in libraries and modules while remaining interoperable with non-promised code? and 2. all of the necessary closure generation leads to a lot of opportunities for deoptimization and impaired performance. Promises/A+ is the best attempt so far to get something like this up and running in pure ES5.

Reply?

You did a good job at introducing the abstraction. I think it's important to point out that future-s are monads. This is is an important fact that should be reflected in the API, as discussed here.

Reply?

I do not understand, though, why is it part of the DOM and not simply part of ECMAScript itself? This is really unrelated to DOM, it is a basic concept within the entire language (and I guess would also be used in much more places than the DOM).

Reply?

I believe they are planned to be part of some future¹ version of ES, but the problem is they are needed for web APIs now, not when ES7 rolls around. Otherwise, W3C working groups will keep producing poorer, event based APIs, or reinventing promises. At least this way the W3C has a single, well-designed version of them that can be used throughout its APIs. If they can eventually be made part of the language, so much the better.

¹ No pun intended.

Reply?

http://www.xanthir.com/b4PV1#4PV1-6

Reply?

Does anyone know any examples of experimental languages that do something like this?

E http://erights.org , which was the first language with promises (aka "futures") with the virtues you explain, as well as an integrated syntax ( http://www.erights.org/talks/promises/paper/tgc05.pdf ). JS promises derive through at least two chains from E promises:

E -> Python Deferreds -> MochiKit Deferreds -> Dojo Deferreds -> Promises/A -> Promises/A+ E -> Waterken Q -> Caja Q -> Kris Kowal's Q -> Promises/C -> Promises/A+ (Q, Promises/A+) -> DOMFutures

Reply?

Formatting not what I intended. Trying again

E -> Python Deferreds -> MochiKit Deferreds -> Dojo Deferreds -> Promises/A -> Promises/A+

E -> Waterken Q -> Caja Q -> Kris Kowal's Q -> Promises/C -> Promises/A+

(Q, Promises/A+) -> DOMFutures

Reply?