25 — Space

Tuesday, 07 April 2020

Presenter Neil Resnik

BJ’s recording

A machine needs space to store the data needed for the evaluation of a program. The execution of instructions has different needs for different kinds of data. Let’s analyze where the data lives in our machine. Then we assess the kinds of space needed, and how managing storage works.

Which Instructions Need Space

Before we focus on the organization of the store itself, let’s briefly look at how the execution of instructions on the CESK machine consumes space.

Stop! Take a close look at figures 90 and 91

When does the environment grow? The machine adds a variable binding whenever the right-hand side of a decl is evaluated and the machine switches the execution of the decl’s body.

The environment never shrinks but the E register gets sometimes set to the empty environment, which may make certain branches of the environment tree unreachable.

When does the stack grow? The stack grows every time we evaluate the arguments of an application of a primitive function. It also grows when we evaluate the right-hand side of a decl. This second form of growth also tells us when the stack would grow for function applications:

When does the stack shrink? The stack shrinks when the sub-expressions to function have been evaluated.

In our extremely small language, we don’t really have functions, we don’t have unlimited allocation of memory in any way whatsoever. But, if we imagine dynamic processes such as potentially recursive functions, mutually connected methods, while loops, etc., it is easy to see that we need to discuss where stack and store space is consumed.

Dybvig has demonstrated that such stack implementation on existing hardware stack cost one bit per frame and a word per finite stack segment. Although this cost isn’t zero, it is extremely low.

In particular, we cannot predict how deep arguments are nested in a language that permits cyclic references among its active entities (functions, methods). So while stacks have to be finite, they must be growable to arbitrary size. If a language supports "grab", this idea is a given. Other languages struggle to get this right.

Onward to stores. The store grows every time we evaluate an 'alloc expression on two values. Due to “old” jargon, we often speak of heap allocation. Programmers who use C-style languages still use this language, and from there it has leaped into the Java world. In addition to low-level alloc and malloc, high-level languages allocate memory in the store for lambda (pairing the code with the environment), new for the creation of objects, cons for constructing lists, and so on.

When does the store shrink? Not so far, and because dynamic programs allocate a lot of space in the store, we must figure out some means for shrinking it.

Where does All this Data Live?

Tracing

What we just described—implicitly—are the first three steps of reaching all locations that might possibly be useful. It is time to describe the algorithm.

The standard algorithm for finding the set of usefull locations “colors” the store’s locations “black,” “gray,” or “white.” A black location is live, and the pair at this location references only black and gray locations. In contrast, a gray location is useful, but the pair at that location might reference a white place in the store. Finally, a white location contains a potentially useless value. At the end of the coloring process, no locations are gray, and all white ones are garbage.

Stop! Explain each of these four statements.

Figure 93: Garbage Collection in Process

Operationally this description translates into a store that has only two cells colored gray: see the first store in ~figure-ref{fig:gc-1}. The collector picks those, colors them black, and leaves everything else alone because locations 10 and 11 contain only plain integers. The fwd label in 10 tells us that this pair will survive and the 0 in 11 show us in which location the pair will end up. Finally, the last diagram shows the store after the garbage collection is over. It contains only one pair—meaning 10 more locations are available to complete the program.

Figure 94: Garbage Collection in Process

At this point in the execution, the store is a regular graph with numerous cross connections and even cycles. Take a look at the first store in figure 94. Every even-numbered location points to the right, because a program may reach this location via '(_ dot right) and '(_ set-right _). Every odd-numbered location contains a location, marked as [L n] for some number n. Graphically these locations are related to the left-most neighbor of the pair.

Figure 94 also shows two gray pairs in this first store: our gray locations. The pair at location 10 is reachable from both the C register and the environments in the E and K registers. The pair at 8 is only reachable from E.

The six stores below the first in figure 94 illustrate how the garbage collector works it way through the store. Our collector starts with the rightmost pair, “forwards” it to the black pool, and thus marks it as saved. Then the algorithm progressively traverses the store from the right to the left until all locations have been painted black.

Stop! What does this mean?

The final store in figure 94 shows how the collector re-allocated pairs as they were marked as survivors. In this store, the pairs are allocated in reverse order from the very first store. The relationships among them are the same: the right side of each pair points to the same pair as in the first store.

However, the collector filled the entire store again with the surviving pairs—meaning there is no space for the remaining pair. In this situation, some runtime systems error out with strange actions, some others manage to signal an error to the programmer.

Implementation

The addition of a garbage collector causes a linguistic dilemma, namely, what to call the program that actually uses the space in a store. In our case it is a CESK machine; if we had CPS transformed our interpreter, the space-consuming program could be an interpreter. But, this space consuming program doesn’t have to be one of those special programs. When a compiler translates a source program to a target program, the garbage collector collaborates with an arbitrary program—the program that the developer handed to the compiler. So, for historic reasons, we refer to the space-consuming program as the mutator.

Figure 95: Mutator and Memory Manager

Lectures/25/find-roots.rkt

#lang racket

;; Find the roots in the registers of a C E S K machine

(provide

;; for pedagogic reasons, we tag them differently from locs:

(struct-out root)

#; {Exp Env Store Stack -> [Listof root]}

find-roots)

;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

(require "show-with-store.rkt")

(require "locs-and-roots.rkt")

(require "../21/while.rkt")

;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

(define (find-roots C E S K)

(define lst (append (find-in-C C) (find-in-E E) (find-in-K K)))

(set->list (apply set lst)))

#; {Control -> [Listof root]}

(define (find-in-C cc)

(match cc

[(? root? l) (list cc)]

[(list (? root? L) o (? root? R)) (list L R)]

[(list (? root? L) o _) (list L)]

[(list _ o (? root? R)) (list R)]

[_ '[]]))

#; {Env -> [Listof root]}

;; ASSUME Env = [Listof [List Var Value]]

(define (find-in-E env [vars-seen '()])

(cond

[(empty? env) '()]

[else

(match-define (list x val) (first env))

(if (or (member x vars-seen) (number? val))

(find-in-E (rest env) (cons x vars-seen))

(cons val (find-in-E (rest env) (cons x vars-seen))))]))

#; {Stack -> [Listof root]}

(define (find-in-K stack)

(append-map find-Frame-roots stack))

#; {Frame -> [Listof root]}

;; ASSUME all frames contain the environment in the same spot

(define (find-Frame-roots frame)

(match frame

[(list _tag env _op possibly-root)

(define roots (find-in-E env))

(if (root? possibly-root) (cons possibly-root roots) roots)]))

Figure 96: Roots

Lectures/25/heaps.rkt

#lang racket

(provide

SPACE ;; the number of locations in a store

SIZE  ;; size of allocated blocks

#; {type Heap}

(struct-out heap)

#; {[Listof Location] -> Heap}

make-plain)

;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

(define SPACE 12)

(define SIZE 2)

(struct heap

[ store             ;; [Vectorof Value] : actual storage

secondary         ;; [Vectorof Value] : swap-in storage

{next #:mutable}  ;; Nat : next place to place value into `store`

roots             ;; [-> (Listof Location)] : find root locations

] #:transparent)

(define (make-plain (find-roots (lambda () '[])))

(heap (make-vector SPACE #f) (make-vector SPACE #f) 0 find-roots))

Figure 97: Laying Out the Stores

The API of the store remains the same but the implementation of the three functions changes slightly; see figure 98. Indeed, the advertised API (see provide) has always just stated that there is a kind of value called Store without detailing what it is. Internally the new component uses Heaps as stores. The three exported functions—alloc, retrieve, update—mutate the primary store and the next field in the heap in a straightforward manner.

The key function is an internal one: alloc-proper. It first checks whether there is enough space left in the primary store. If so, it places the given values into the primary storage of the given heap and updates the next field appropriately; the result is the location of this new block of values. If there isn’t enough space, it signals this fact with #false.

Lectures/25/store-finite.rkt

#lang racket

(provide

;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

#; {type Store}

#; {[-> [Listof Location]] -> Store}

;; create (initialize) a store

;; EFFECT squirrel away a function that delivers a list of root locs

make-plain

#; {Store [Listof Value : SIZE] ->* Location Store}

alloc

#; {Store Location -> Value}

retrieve

#; {Store Location Value -> Store}

update)

;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

(require "locs-and-roots.rkt")

(require "heaps.rkt")

(require "collect.rkt")

;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

#; {type Store ~ Heap}

(define (alloc h vals)

(define new (alloc-proper h vals))

(cond

[new (values (loc new) h)]

[else

(displayln '[collecting garbage])

(define g (collect h))

(define new2 (alloc-proper g vals))

(values (loc (or new2 (error 'alloc "out of space"))) g)]))

#; {Heap [Listof Value : SIZE] -> (U #false Location)}

(define (alloc-proper h vals)

(cond

[(<= (+ (heap-next h) SIZE) SPACE)

(begin0 (heap-next h)

(for ([v vals])

(vector-set! (heap-store h) (heap-next h) v)

(set-heap-next! h (+ (heap-next h) 1))))]

[else #false]))

(define (retrieve s loc)

(vector-ref (heap-store s) (loc-ref loc)))

(define (update s loc val)

(vector-set! (heap-store s) (loc-ref loc) val)

s)

Figure 98: The Store

When alloc finds out that it is not possible to place the given values into the primary storage, it calls collect. This call yields a new heap in which the primary and secondary storage parts have switched roles. The content of the useful locations—the black ones—is in the primary storage; the next field is set to where new space—if any can be found. Hence, alloc can now try a second time to place the given values into the store. If this second attempt—which we know from figure 94 is a distinct possibility—the function signals a fatal error.

At last, we can inspect the garbage collector itself; see figure 99.

Lectures/25/collect.rkt

#lang racket

(provide

#; {Heap -> Heap}

;; collect the garbage and switch the two stores in the heap

collect)

;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

(require "heaps.rkt")

(require "locs-and-roots.rkt")

(require "store-list.rkt")

(require "../21/while.rkt")

;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

(define VISITED 'fwd) ;; the "black" location marker

(define (collect h)

(match-define  (heap store secondary _ *find-roots) h)

(define nuheap (heap secondary store 0 *find-roots))

(define roots0 (*find-roots))

(define *roots (roots->locs roots0))

(while (cons? *roots) do

;; pick a gray locatuon

(define the-gray (loc-ref (first *roots)))

(set! *roots (rest *roots))

(unless (eq? (vector-ref store the-gray) VISITED)

(for* ((i (in-range SIZE))

[val (in-value (vector-ref store (+ the-gray i)))])

;; re-allocate in `secondary`

(vector-set! secondary (heap-next nuheap) val)

(set-heap-next! nuheap (+ (heap-next nuheap) 1))

;; identify additional gray locations

(when (loc? val) (set! *roots (cons val *roots))))

;; paint `the-gray` black

(vector-set! store (+ the-gray 1) (- (heap-next nuheap) SIZE))

(vector-set! store the-gray VISITED)))

;; adjust locations in `secondary`;  all necessary values are moved

(for* ([i (in-range (heap-next nuheap))]

[val (in-value (vector-ref secondary i))]

#:when (loc? val))

(vector-set! secondary i (loc (forward store (loc-ref val)))))

;; updates root cells in mutator

(for ([r roots0])

(set-root-ref! r (forward store (root-ref r))))

nuheap)

(define (forward store place) (vector-ref store (+ place 1)))

(define (roots->locs r*) (map (λ (r) (loc (root-ref r))) r*))

Figure 99: A Garbage Collector

Embarrassing Pauses

A tracing garbage collector stops the mutator—here our CESK machine—from executing any code. It must do so because it moves values to which the mutator refers; if the mutator were to refer to a location in the old heap arrangement, it might accidentally retrieve a garbage value such as VISITED.

This stop-and-pause the mutator action may lead to so-called “embarrassing pauses,” a time when some program seemingly does nothing. While modern collectors have reduced the duration of these pauses to an often tolerable span, all of you have encountered these pauses—either in Fundamentals I when you noticed the green recycle icon spinning in DrRacket or in web browsers when things just took a tad longer than you expected.

These pauses and other aspects of garbage collection keep researchers in this field on their toes, and the area remains rather active.