29 — Continuations

Tuesday, 21 April 2020

BJ’s recording

At any point during a program execution, we can look at the history of the process, its current state, and the rest of the computation—the continuation. In some way or another, all programming languages permit the manipulation of the continuation. The most extreme way is to ignore the continuation because the current state satisfies some attribute; calling exit or raising an uncatchable exception are concrete means for accomplishing this goal.

Some programming languages provide programmatic access to continuations. Scheme programs, for example, may turn it into a function; so does Racket as a descendant of Scheme. Dialects of ML (SML/NJ, OCaml) supply a continuation object on demand, to which programs can throw values; Smalltalk and Squeak call it a stack object (I think).

Unsurprisingly, the power of manipulating continuations as ordinary value is extreme (though the cost isn’t low). With the availability of continuation objects, software developers can create any kind of control pattern that the language designer deemed unworthy pf a special construct: Python-style generators, backtracking, intelligent search, context switching, relating event handlers to procedural libraries, time-preempted encapsulation, process management, and more.

You may think that these topics sound rather esoteric. But it turns out that ordinary consumers of software started operating this way in the late 1990s, with the advent of the web. People would open several browser windows for the same site, say for booking flights, to compare them. They would switch back and forth to explore details. And then they would decide to purchase one—only to Queinnec was the first to point out this tight connection; figure 1 here describes the problem with a diagram and triggered the first class-action suit in this arena. get the wrong item. It turns out that server-based interactions written in languages without continuations could not cope with these manipulations of continuations as in general developers failed to understand the difference between environments (web page content), stores (cookies and databases), and process progress (continuations). The resulting software was error-prone and caused many lawsuits.

The goal of this lecture is to provide a glimpse at the expressive power of continuation objects. We will start with simple examples, progress to Python-style generators as an intermediate point, and finish up with the core of a process management component in the operating system. Indeed, this idea of managing computational processes in the operating system—and elsewhere—is the motivational idea that we start with.

Sharing Resources, Sharing Time

The operating systems you use on your phones and laptops enable you to run several different programs at once, many more than there are CPUs or Cores in your hardware. This “simultaneous” solution is thus an illusion. In reality the operating systems switches so quickly between different tasks that you think all of them serve you, the master of the computer, at the same time. While this illusion is just an example, it perfectly illustrates the task that an operating system performs:

No matter how “wealthy” our computers become, some computational resource (time, energy, storage memory, printers, network access, etc) will always be scarce. It is the task of the operating system to play arbiter—market in economic terms—to match demand for a resource with supply.

But of all tasks, an operating system’s primary task is to manage processes and their access to the cpu. Continuation programming is the best way to understand how this works.

The Big Picture

The dispatcher is the first small piece of software that exists. It uses its time slice to observe an associated queue. When a process shows up in the queue, the dispatcher sets up this first process to run. A process may spawn other processes, an act that means that a new program is added to the end of the queue. If the currently running process exhausts its allocated slice of time on the CPU before it completes its computation, the dispatcher’s alarm goes off and the rest of the current process’s computation is turned into a new process, which is added to the end of the queue.

The first five rows of the following table illustrate how this phase of process-dispatching works:





the dispatcher exists, Process 0 sows up in the queue





the dispatcher sets the clock and permits Process 0 to run





Process 0 spawns Process 1, which gets added to the end of the queue





Process 0 does not complete in the allotted time, the dispatcher swaps it out, adding the rest of the computation to the end of the queue





the spawned Process 1 is given time on the CPU/core





eventually many processes are in the queue with one running at any moment

The last row is suggestive of how the queue eventually fills up.

Processes cannot only spawn another process, they may also suspend themselves because there is no work to do; they can request that other processes suspend themselves or even kill themselves.Do read up on why the Java group deprecated the Thread.stop procedure.

At first glance, an operating system uses a generator-style yield construct to switch back and forth between its driver and the processes. Except that time seems to play a role, and time does not play a role in the generators you know.

Using Racket

This lecture uses plain Racket instead of the model languages of the past. Thus,

instead of


we write






a variable definition






a variable reference



if plus predicates

One final major difference to keep in mind is the guaranteed left-to-right evaluation of the expressions in a function application, including those for functions from the prelude (+, *, etc).

Small Examples

Figure 113 collects some “finger exercises” that we considered in the context of Toy, with "grab" instead of let/cc.


  #lang racket
  ;; - - - - - - - - - - - - - - - - - - - - -
  ;; what is the rest of the computation: 
  (+ 1
     (let/cc done
       (* 2 (done 42))))
  ;; here is one way to represent it:
  #; (λ (x) (stop (+ 1 x)))
  ;; The stop part reminds us that the reified
  ;; continuation is a bit more than a function:
  ;; it eliminate the rest of the computation
  ;; at its own call site. 
  ;; - - - - - - - - - - - - - - - - - - - - -
   (let/cc exit1
     (* 2
        (let/cc done
          (+ 1 (exit1 42)))))
  ;; - - - - - - - - - - - - - - - - - - - - - 
   (let/cc exit1
     (* 2
        (let/cc done
          (+ (done 1) (exit1 42)))))
  ;; - - - - - - - - - - - - - - - - - - - - -
  (let ([janus (λ (x) x)])
    (set! janus
          (let/cc flip-janus
            (if (and (number? janus) (zero? janus))
    (if (and (number? janus) (zero? janus))
        (* 666 [janus 0])))

Figure 113: Simple Examples

Stop! Work through them to practice reasoning with let/cc. With “work thru” we mean figure out two pieces of information per program:
  • What is the rest of the computation for each let/cc expression?

  • What is the rest of the computation that every call of a continuation eliminates?

Some expressions come with some answers. Figuring out the remaining answers will strengthen your sense of continuation grabbing and continuation calling.

Stop! Here is the answer for the last example:
; flip-janus can be represented as:
(λ (result-of-letcc-done)
   (set! cell result-of-letcc-done)
   (if (and (number? cell) (zero? cell))
       [janus 0])))
The difference between this example and the first three is that the continuation escapes from the scope set up by let/cc. Then it gets stored in janus, and by calling janus on 0 it changes the value of the variable again.


Racket has a generator library, and it is possible to make an identical implementation and override define. Doing so might obscure the pedagogical point.

For those unfamiliar with these generators, they are roughly like a function. When called a first time, they initialize their control state. Instead of computing one final value, the generator then launches a computation that can be suspended at any point and send a value to the caller. When the generator is called again, it resumes the computation at the point where it was last suspended and continues as before. In principle, generators never terminate. If they do, our implementation raises an exception.

Figure 114 displays a function that approximately acts like a Python-style generator. The definition consists of three pieces:
  • the generator’s name, count-down,

  • a second identifier, suspend-yield, with which the generator produces intermediate values, and

  • the underlying function, which describes the computational process.


  #lang racket
  (provide count-down using-exn)
  ;; - - - - - - - - - - - - - - - - - - - - - - - - - - - 
  (require "gen-implementation.rkt")
  ;; - - - - - - - - - - - - - - - - - - - - - - - - - - - 
  ;; gen yields n, n-1, n-2, ... every time it is called
  ;; until it falls off the end
  (define yield displayln)
  (define count-down-f
    (λ [x] 
      [local ([define pythn
                (λ (x)
                    [(zero? x) using-exn]
                    [else (yield x)
                          (pythn (- x 1))]))])
        (pythn x)]))
  (define/generator count-down suspend-yield
    (λ [x]
      [local ([define pythn
                (λ (x)
                    [(zero? x) using-exn]
                    [else (suspend-yield x)
                          (pythn (- x 1))]))])
        (pythn x)]))
  (define using-exn 111)

Figure 114: A Generator

The purpose of the generator in figure 114 is to produce the numbers n, n-1, ... when given n.

Figure 115 shows how to use this generator. There is an initial call, plus two more with whimsically chosen arguments, because the arguments to the call do not matter.


  #lang racket
  (require "gen-define.rkt")
  ;; - - - - - - - - - - - - - - - - - - - - - - - - - - -
  (define [using n]
    (let* ([x (count-down n)]
           [y (count-down 33)]
           [z (count-down 33)])
      (+ z y x)))
  (module+ test
    (require rackunit)
    (check-equal? (using 3) 6))

Figure 115: Using the Generator of (figure-ref fig:define-generator)

Stop! Since the initial argument is 3, what should be the result of using? What if we call using with 4?


  #lang racket
  ;; - - - - - - - - - - - - - - - - - - - - - - - - - - 
   #; (define/generator g:id name-of-yield:id body:expr)
   ;; SYNTAX
   ;; implements a Python-style generator that is like a
   ;; funciton but `yields` intermediate values 
    (define/generator gen yield pythn)
    (define gen
      (local ((define (yield y)
                (let/cc c
                  (define go resume)
                  (set! resume (enter c))
                  (go y)))
              (define [[enter c] z]
                (let/cc k
                  (set! resume k)
                  (c z)))
              (define resume
                (enter (λ (x) (the-end (pythn x))))))
        (λ (x) [resume x]))))
  (define (the-end x)
    (error 'gen (~a THE-END x)))
  (define THE-END "gen fell off the end ")
  (define gen
      (local ((define [[enter c] z]
                (let/cc k
                  (set! resume k)
                  (c z)))
              (define (yield y)
                (let/cc c
                  (define go resume)
                  (set! resume (enter c))
                  (go y)))
              (define resume
                (enter (λ (x) (the-end ('pythn x))))))
        (λ (x) [resume x])))

Figure 116: Implementing Generators

Finally, figure 116 implements generators in terms of continuations. The define-syntax-rule construct translates the first line, a pattern, into the template expression below; in particular, Racket’s syntax system recognizes the pattern in every module that imports this syntax extension and replaces it with the template after replacing the pattern variables in the template with their matches.

The implementation is based on the following key observation:

The program context and the generator act as partners in a conversation. At any point in time only one partner computes and then yields the right to the other one.

This brings to continuations into play:
  • when the generator computes, our implementation must know the continuation of the client program, that is, the continuation with respect to the last call site of the generator

  • when the client program computes, our implementation must know the continuation of the generator, that is, from which place it will resume its computational process.

Because only of the two continuations is useful at any point in time, we can get away with a single locally defined variable to keep track of them.

The local function definitions serve the purpose of managing the continuations and the back and forth between the two parties. (1) The resume variable stores the continuation of the inactive partner. Its initial value is basically the given function (pythn). With the-end we guarantee that it doesn’t fall off the end.

The enter and yield functions are responsible for keeping track of which continuation to store in resume. The first, enter, consumes a function of one argument and returns a function of one argument. The result grabs the current continuation of the client program, stores it in resume, and calls the given function. The second, yield, consumes a value, grabs the continuation of the generator, retrieves the current value of resume, turns the continuation into the next resume point, and then sends the given value back to the caller—by calling the old content of resume.

A Kernel

If you like to read more about the connection between programming languages and operating systems, let me recommend readings on the Lisp machine, the Smalltalk programming system, and modern resurrections such as The Revenge of the Son of the Lisp Machine and Microsoft’s Singularity Project.

An operating system is a collection of things that don’t fit into a language. There shouldn’t be one. – Dan Ingall, Byte Magazine 1991

Dan Ingall was a member of the team that produced SmallTalk, the very first object-oriented programming language. (Okay, I don’t like Simula67.)

The construction of a process manager assumes the existence of a timer, shown in figure 117. A timer is like a programmable stop watch. We can set it to run for n msec. When it reaches 0, it calls the timer interrupt handler, a function that may inspect and manipulate the state of the currently running process. Finally, a timer can be stopped, in which case no action is taken.—Racket does not support this specific interface, and the rest of the code here is hypothetical. By contrast, similar code will run in Chez Scheme, which does support this kind of interface.

 ; [ {-> Empty} -> Void ]
 ; (set-timer-interrupt-handler ih) runs ih when the timer expires
 ; [ N -> Void ]
 ; (start-timer n) set a timer to expire in n msec
 ; [ -> Void ]

Figure 117: The Assumed Timer Interface

The process manager is presented in figure 118. The module exports two functions: boot and spawn. The boot function consumes a code function. It uses spawn to turn the code function into a process; this act also adds the process to the process *queue. The heart of boot is a driver that inspects the process queue, pulls out the first process, and gives this process a slice of processing time. If the queue is ever empty, driver shuts down.


  #lang racket/gui
   #; { [-> Void] -> Void }
   ;; consumes the code (function) for making an initial process 
   #; { [-> Void] -> Void }
   ;; grants access to processes to spawn more processes 
  ;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
  (require "clock.rkt")
  (require "queue.rkt")
  (require "process.rkt")
  ;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
  (define TICK 5)
  (define *queue (new queue%))
  (define (boot make-process-0)
    (define p0 (spawn make-process-0))
    (let driver ()
        [(send *queue deq!)
         (λ (p) ;; p becomes the current process
           (let/cc back-to-driver
              (make-timer-interrupt-handler p back-to-driver))
             (start-timer TICK)
             (send p run back-to-driver))
        [else '[all done]])))
  #; { [-> Void] -> Void }
  (define (spawn function-of-process)
    (define p
      (new process% [code (λ () (function-of-process) (stop-timer))]))
    (send *queue enq! p)
  #; { Process Continuation  -> Empty }
  (define (make-timer-interrupt-handler p back-to-driver)
    (λ ()
      (let/cc rest-of-process
        (send p swap (lambda () (rest-of-process 'go)))
        (send *queue enq! p)

Figure 118: The Driver

The driver function generalizes the generator implementation in two ways:
  1. It does not rely on the running process to yield control; instead it sets up the timer and uses the timer-interrupt handler to hand control back to the driver.

    Specifically the handler capture the current continuation of the running process, which represents its (control state plus environment and thus access to storage). Then it remembers this continuations in the process itself.

  2. Instead of two partners, the driver scenario deals with many. The driver itself is a party and its continuation is available both to the handler and the run method of the process. Thus no matter whether the process has to be forced out or shuts down, the locus of control can return to driver. The process data representations store the continuations of their computational processes.

Figure 119 demonstrates how to use boot with P0, which in turn starts P1. Both processes here are based on the same code, but abstraction permits separation of action.


  #lang racket
  (require "driver.rkt")
  (define [(make-process action outputs)]
    (let loop ([n outputs])
      (when (cons? n)
        (displayln `[~~~~~~ ,(first n)])
        (loop (rest n))))
    (displayln `[-- ,(string-join (map ~a outputs) ", ") --]))
  (define P0 (make-process (λ () (spawn P1)) (build-list 10 -)))
  (define P1 (make-process void (build-list 10 +)))
  (boot P0)

Figure 119: Booting the Driver

A process data representation is an object with two methods. The run method accepts a return continuation, which is called in case the underlying code runs to completion. The swap method updates the control state of the process, that is, what is left to do when it gets another chance to run. See figure 120 for details.


  #lang racket
   #; { Class [init (-> Void)]
              (show (->m JSexpr))
              (run  (->m Continuation))
              (swap (->m Continuation)) }
  ;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
  (define process%
    (let* ([*pid 0]
           [pid+ (λ () (begin0 *pid (set! *pid (+ *pid 1))))])
      (class object% [init code]
        (field [thunk  (lambda () (code) (return))])
        (field [return #false])
        (field [pid (pid+)])
        (define/public (show)
          `[process ,pid])
        (define/public (run back)
          (set! return back)
        (define/public (swap t)
          (set! thunk t)))))

Figure 120: A Process Representation


The queue implementation is entirely standard.


  #lang racket
   #;{ (All (X)
             [enq! (->m X Void)]
             (deq! (->m (U X False))))) }
  ;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
  (define queue%
    (class object%
      (define *q '[])
      #; {-> JSexpr}
      (define/public (show)
        `[queue ,(map (λ (i) (send i show)) *q)])
      #; {X/show -> Void}
      (define/public (enq! p)
        (set! *q (append *q (list p))))
      #; {-> (X/show u False)}
      (define/public (deq!)
        (and (cons? *q) (begin0 (first *q) (set! *q (rest *q)))))))

Figure 121: A Queue