23 — CEK, CESK

Tuesday, 31 March 2020

Presenters (1) Kaylin Devchand & Kate Rupa (2) Sean Wallace, Zachary Wolfe

BJ’s recording

Our next step is to re-introduce variables. First we deal with variables à la mathematics and then with assignable ones.

CEK

We know from our work with interpreters that variable expressions must be interpreted within the scopes of all of its variable occurrences.

An expression is represented with abstract syntax trees, and a scope is represented with an environment. Even if we ignore the exact nature of environments we can still equip the existing machine rules for ArithmeticExpr with an environment register; see figure 87. The side conditions on these expression are as before, which is why they are omitted.

current						next
C		E		K		C		E		K
(e1 o n2)		E		K		e1		E		K, (L E o n2)
(e1 o e2)		E		K		e2		E		K, (R E o e1)
(n1 o n2)		E		K		n		∅		K
n		∅		K, (L E o n1)		(n o n1)		E		K
n		∅		K, (R E o e1)		(e1 o n)		E		K

Figure 87: A Tabular Description of the CEK Machine

Thus far the environment was just along for the ride, without being used in any way. Clearly, before the machine can use the environment to find the value of variables, such values need to get into the environment. And this can happen only when the machine evaluates a decl tree. Specifically, the machine must evaluate the right-hand side sub-expression and remember to put its value into the environment; after that happens, it may interpret the body sub-expression of the decl tree.

current

next

C

E

K

C

E

K

if

(decl x e1 e)

E

K

e1

E

K, (D E x e)

n

∅

K, (D E x e)

e

E, (x ↦ n)

K

x

E

K

v

∅

K

(x ↦ v) ∈ E

Figure 88: The CEK Machine Transitions for Vars and Decls

Added post lecture

The Nature of CEK States

In general, the environment register must provide meaning—a value—for all variable references without corresponding declaration in the code register. Environments in a stack frame explain the meaning of the variable references without decl in its expressions.

Stop! How can we get back to a form of CK machine?

Stack vs Tree

In the previous lecture we observed that we can use a stack to represent the rest of the instructions, that is, those instructions that must be executed when the instructions in the control register are all done. This remains the case for the language with declared variables.

At first glance, the content of the environment register seems to only grow; at least only one transition rule adds (or replaces) an (existing) association of a variable with a value. But this impression is wrong because (1) a return state wipes out the current register and (2) every time the machine pops a frame from the stack register, it extracts an environment E and sticks it into the environment register.

Nevertheless, with only decl in the language, the expression is evaluated exactly once and we know that it is evaluated in the context of its original stack. Hence, one could “optimize” this machine and place variable bindings onto this stack until the instruction in decl’s body are all run. At that point, it would be safe to pop the variable binding from the stack.

This observation was exploited in the early days of compiler work. Once a stack was accepted (which took two decades), it was used to represent both the control context and the variable context. Indeed, language designers turned this idea around and rejected language features for a long time if they destroyed this unity of control and variable information.

Procedures are one critical example of this kind. They—not the closures or objects found in modern languages—were added to languages in a rather restricted manner. C is one such traditional language, and Rust has in some ways inherited this idea.

Adding closures or objects destroys this unity and demands an environment representation that resembles a tree.

Adding First-Class Functions

Stop! Work out the details on paper and pencil.

Implementing the CEK Machine

The code in figure 89 uses the same environment that we always used. Otherwise it is a transliteration of the transition rules into the usual code schema.

Lectures/23/cek.rkt

#lang racket

(require "../21/while.rkt" "../22/stack.rkt")

(require "../6/environment.rkt")

(require "simple-show.rkt")

;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

(define LEFT "left")

(define RGHT "right")

(define DECL "let")

(struct decl [var rhs body] #:transparent)

#; {type VarExpr = ... || [decl Var VarExpr VarExpr] || Var}

#; {type Var = Symbol}

(define var? symbol?)

#; {type Env : mapping Var to Val}

#; {type Val = Number}

(define val? number?)

#; {VarExpr -> Number}

;; print each step of the calculation that reduces `expr` to a number

(define (driver initial)

(define-values (*C *E *K) (load initial))

(show-state *C *E *K)

(while (not (final? *C *E *K)) do

(set!-values (*C *E *K) (transition *C *E *K))

(show-state *C *E *K))

(unload *C *E *K))

#; {VarExpr Env Stack -> VarExpr Env Stack}

(define (transition C E K)

(match* { C E K }

[{(? var?) E stack}

(define val (lookup C E))

(show-return-state val '[] stack)

(values val '[] stack)]

[{(decl x ae bdy) E stack }

(values ae E (push stack (list DECL x bdy E)))]

[{(? val? n) E (app pop `((,(? (is? DECL)) ,x ,bdy ,E-d) ,K))}

(values bdy (add x n E-d) K)]

[{(list(? not-number? ae_1) o (? number? ae_2)) E stack}

(values ae_1 E (push stack (list LEFT o ae_2 E)))]

[{(list ae_1 o (? not-number? ae_2)) E stack }

(values ae_2 E (push stack (list RGHT o ae_1 E)))]

[{(list (? number? l) o (? number? r)) E stack }

(define n (reduce C))

(show-return-state n E stack)

(values n mt stack)]

[{(? val? n) E (app pop `((,(? (is? RGHT)) ,o ,ae_1 ,E-r) ,K))}

(values (list ae_1 o n) E-r K)]

[{(? val? n) E (app pop `((,(? (is? LEFT)) ,o ,ae_2 ,E-l) ,K))}

(values (list n o ae_2) E-l K)]))

#; {VarExpr -> VarExpr Env Stack}

(define (load ae) (values ae '[] mt))

#; {VarExpr Env Stack -> Number}

(define (unload n _1 _2) n)

#; {VarExpr Env Stack -> Boolean}

(define (final? control E stack)

(and (val? control) (equal? '[] E) (equal? mt stack)))

#; {(list Number o Number) -> Number}

;; okay this is a trick, but almost every language has this trick

(define ns (make-base-namespace))

(define (reduce ae)

( (eval (second ae) ns) (first ae) (third ae)))

(define not-number? (compose not number?))

;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

(provide driver decl)

Figure 89: Implementing the CEK Machine

CESK

Adding assignments to VarExpr does in principle not require a store. Every decl is evaluated exactly once. Hence, if we systematically rename variables (modern parlance “refactor”) variables before evaluation, we can make sure that no variable name is ever in scope twice. Updating the environment directly would suffice; indeed, we could get away with updating it functionally.

The point of a model is to express realistic ideas with minimal effort. We know that first-class functions or objects would demand the separation of environment and store. Instead of those, however, we pick allocated structures as representative and yet slightly simpler linguistic constructs that demand such a separation. As we proceed, imagine the presence of the complicated ones, too.

Semantically, alloc puts two values into the store. The two are treated as a pair. The dot expressions retrieve the left or right part of such a pair. The set... expressions modify the left or right part of the pair. The setters return the old values of the fields of a pair that they modify.

As above, we first introduce stores into the machine; see figure 90 So far, the store is simply along for the ride.

current

next

C

E

S

K

C

E

S

K

(e1 o v2)

E

S

K

e1

E

S

K, (L E o v2)

(e1 o e2)

E

S

K

e2

E

S

K, (R E o e1)

(n1 o n2)

E

S

K

n

∅

S

K

v

∅

S

K, (L E o v1)

(v o v1)

E

S

K

v

∅

S

K, (R E o e1)

(e1 o v)

E

S

K

(decl x e1 e)

E

S

K

e1

E

S

K, (D E x e)

v

∅

S

K, (D E x e)

e

E, (x ↦ v)

S

K

x

E

S

K

v

∅

S

K

where (x ↦ v) ∈ E

Figure 90: A Tabular Description of the CESK Machine

But, once we have a machine with a store, we can use it to interpret the additional “arithmetic” operations in Expr; see figure 91.

current

next

C

E

S

K

C

E

S

K

(v1 alloc v2)

E

S

K

l

∅

S, (l ↦ v1, l+1 ↦ v2)

K

where l and l+1 are not in S

(l dot left)

E

S

K

v

∅

S

K

where (l ↦ v) ∈ S

(l dot right)

E

S

K

v

∅

S

K

where (l+1 ↦ v) ∈ S

(l setleft v1)

E

S

K

v

∅

S, (l ↦ v1)

K

where (l ↦ v) ∈ S

(l setright v1)

E

S

K

v

∅

S, (l+1 ↦ v1)

K

where (l ↦ v) ∈ S

Figure 91: The CESK Machine Transitions for Mutable Pairs

The notation l+1 suggests that locations are numbers, but this is not necessarily so. In some languages this is the case. If such languages also lack (type) soundness, it is even possible to “confuse” numbers that represent locations with actual numbers, multiply or divide two such locations (meaningless operations!), and get into all kind of trouble. C, C++, and Objective C are still-prominent languages that take this view.

Stop! Is the CESK interpretation of Expr sound?

Wait! There are no types, what could “type sound” mean here? In the absence of types, sound means the “interpretation” of all programs has an expected outcome: a result, a loop, or an erroneous situation that we accept, such as not being able to add a number to a function. And this is the hint.

Can the machine produce nonsensical results? Should it get stuck in some states and doesn’t even though it’s neither final nor really defined for the current state?

This is one of two topics we will discuss next time.

Implementing the CESK Machine

Figure 92 displays a fairly standard implementation. The key difference between the CESK machine and the CEK machine concerns the transitions for alloc, dot left, dot right, setleft and setright. Some fit the standard format of infix arithmetic notation (intentionally so); the one for dot notation doesn’t and therefore needs two special-case transitions.

Lectures/23/cesk.rkt

#lang racket

(require "store.rkt")

(require "../21/while.rkt" "../22/stack.rkt" "../6/environment.rkt")

(require "show-with-store.rkt")

;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

(define LEFT  "left")

(define RGHT "right")

(define DECL  "let")

(struct decl [var rhs body] #:transparent)

#; {type Expr = ...

|| [Expr alloc Exp]

|| [Expr dot left]

|| [Expr dot right]

|| [Expr setleft Expr]

|| [Expr setright Expr]}

#; {type Var = Symbol}

(define var? symbol?)

#; {type Env mapping Var to Val}

#; {type StacK = [Listof Frame]}

#; {type Frame = [list {LEFT || RIGHT || DECL} Env X Expr]}

#; {type Loc}

#; {type Store mapping Locs to Val}

#; {type Val = Number || Loc}

(define (val? x) (or (number? x) (loc? x)))

#; {Expr -> Number}

;; print each step of the calculation that reduces `expr` to a number

(define (driver initial)

(define-values (*C *E *S *K) (load initial))

(show-state *C *E *S *K)

(with-handlers ([exn:fail? (λ (xn) (show-core-dump xn *C *E *S *K))])

(while (not (final? *C *E *S *K)) do

(set!-values (*C *E *S *K) (transition *C *E *S *K))

(show-state *C *E *S *K))

(unload *C *E *S *K)))

#; {Expr Env Store Stack -> Expr Env Store Stack}

(define (transition C E S K)

(match* { C E S K }

;; - - - special case for 'dot [left | right] - - - - - - - - -

[{(list (? val? v) 'dot (or 'left 'right))  E S K}

(define-values (n S+) (reduce C S))

(values n empty S+ K)]

[{(list ae_1 'dot (and which (or 'left 'right))) E S K }

(values ae_1 E S (push K (list LEFT E 'dot which)))]

;; - - - because left and right are not expressions - - - - - -

[{(? var?) E S K }

(define val (lookup C E))

(values val empty S K)]

[{(decl x ae bdy) E S K }

(values ae E S (push K (list DECL E x bdy)))]

[{(? val? n) E S (app pop `((,(? (is? DECL)) ,E-d ,x ,bdy) ,K))}

(values bdy (add x n E-d) S K)]

[{(list (? not-val? ae_1) o (? val? v)) E S K}

(values ae_1 E S (push K (list LEFT E o v)))]

[{(list ae_1 o (? not-val? ae_2)) E S K }

(values ae_2 E S (push K (list RGHT E o ae_1)))]

[{(list (? val? l) o (? val? r)) E S K }

(define-values (n S+) (reduce C S))

(values n empty S+ K)]

[{(? val? v) E S (app pop `((,(? (is? RGHT)) ,E-r ,o ,ae_1) ,K))}

(values (list ae_1 o v) E-r S K)]

[{(? val? v) E S (app pop `((,(? (is? LEFT)) ,E-l ,o ,ae_2) ,K))}

(values (list v o ae_2) E-l S K)]))

#; {Expr -> Expr Env Store Stack}

(define (load ae) (values ae '[] plain mt))

#; {Expr Env Store Stack -> Number}

(define (unload n _1 S _3)

(if (loc? n) (retrieve-pair S n) n))

#; {Expr Env Store Stack -> Boolean}

(define (final? control E S K)

(and (val? control) (equal? mt K)))

#; {(list Number o Number) Store -> Number Store}

(define ns (make-base-namespace))

(define (reduce ae S)

(match ae

[(list val1 'alloc val2) (alloc S val1 val2)]

[(list l 'set-left  v) (values (retrieve S l)

(update S l v))]

[(list l 'set-right v) (values (retrieve S (loc+1 l))

(update S (loc+1 l) v))]

[(list l 'set sel nu)  (error 'set "")]

[(list l 'dot 'left)  (values (retrieve S l) S)]

[(list l 'dot 'right) (values (retrieve S (loc+1 l)) S)]

[(list l 'dot sel)           (error 'dot "")]

;; rely on meta-language for arithmetic

[(list lft o rgt) (values ((eval o ns) lft rgt) S)]))

(define not-val? (compose not val?))

;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

(provide driver decl)

Figure 92: Implementing the CESK Machine