5 Racket Internalizes Extra-Linguistic Mechanisms

While many programming problems originate in a “real” world, program development is also a problem domain. As such, tools that support programming deserve a language of their own. Compiler writers take this idea seriously; for example, Dybvig and his group have developed a language for stating compilers as nano-scale transformations and used it for both educational (Sarkar et al. 2005) and commercial purposes (Keep and Dybvig 2013). When it comes to program development or program execution, however, IDEs resort to mechanisms from the surrounding operating system. They force programmers to develop programs in project contexts, delegate program execution to operating systems, and use external tools to inspect programs and their execution states. Racket’s focus on languages as the key to problem solving points to the alternative solution of turning these extra-linguistic mechanisms into linguistic constructs (Flatt et al. 1999).This guideline does not mean that linguistic constructs ought to replace all tools. It merely means that the creation of every tool must raise the question whether it should be a tool or a part of the language. Some tasks are best left to tools, others are better realized as extensions of the language. Finding a good balance is a difficult problem. For example, the creator of a Racket library or a large system may very well wish to think in terms of projects, but a novice student must never have to say “create project” before writing and running a “hello world” program.

To appreciate this idea, consider the original problem of building a pedagogic IDE for novice programmers. Clearly, the emphasis on pedagogy and novices prohibits the use of “projects;” students should be able to type in programs without any knowledge about computers and to run these programs without leaving the IDE they use. By implication, the IDE runs student programs under its control. Students make mistakes, though, and one common mistake is to launch a diverging program, that is, a program that consumes unbounded amounts of time, memory, or other resources (e.g., file ports, database handles, network connections; sometimes the access may be via instructor-provided libraries). Similarly, novices want to find mistakes in programs, meaning their instructors want to show them how to step through a program’s execution. Finally, when a student submits a program to some homework server, this program must run in a security context that prohibits it from inspecting other students’ solutions, attacking the server, and so on.

A close look at these requirements immediately suggests several areas of concern. Due to its design feedback loop, Racket includes the following external mechanisms as constructs at the moment: inspectors, which establish a hierarchy of access rights; threads that can be shut down from the outside; sandboxes, which restrict access to services; custodians, which manage file handles, sockets, and database connections; eventspaces, which deal with GUI resources and events; and several more. The remainder of this section sketches two of these capabilities—inspectors and custodians—and how providing them inside the language provides fine-grained control over inspection and resources.

inspector.rkt

#lang racket/base

(define the-inspector (current-inspector))

(define sub-inspector (make-inspector the-inspector))

(define v

(parameterize ([current-inspector sub-inspector])

(dynamic-require "inspected.rkt" 'instance-of-s)))

Figure 8: Inspection in Racket

inspected.rkt

#lang racket/base

(provide instance-of-s)

(struct s (fld))

(define instance-of-s (s 42))

Figure 9: Inspection in Racket

Figure 8 and~figure 9 demonstrate how Racket turns program inspection into a linguistic construct. Ordinarily, a Racket structure declaration like the one for s in figure 9 defines several functions: a constructor s, a field accessor s-fld, and a predicate s?. Unless a module exports the field accessor, instances of s are opaque to other modules in the system, i.e., other modules cannot view, access, or mutate the content of field fld in an instance. For example, dynamically loading module inspected.rkt from figure 9, retrieving instance-of-s, and printing it would reveal no information:

To address these needs, Racket evaluates modules under a hierarchy of inspectors. If two modules run under the same inspector or incomparable inspectors in the hierarchy, they cannot view, access, or mutate each others structures unless they explicitly grant these rights via provides of the respective functions. In contrast, if module A runs under the control of inspector i and another module B runs under the control of an inspector j that is below i, A can inspect B’s structures—whether B grants these rights or not.

universe.rkt

#lang racket

; [-> X1] ... [-> Xn] ->* X1 ... Xn

; run the given thunks as parallel threads and produce their results

; effect: propagate the exceptions that they raise

(define (launch-many-worlds* . th*)

; allocate resources of th ... in the currently active custodian

(define cc (current-custodian))

; allocate resources of launch-many-worlds in new custodian c*

(define c* (make-custodian))

(define ch (make-channel))

(parameterize ([current-custodian c*])

...

(channel-put ch

(list i (parameterize ([current-custodian cc]) (th)))))

; th ... send values to channel ch

; if any of these is an exception structure, shut down

...

(when (exn? x)

(custodian-shutdown-all c*)

(raise x))

...)

Figure 10: Programming operating-systems patterns in Racket

Figure 10 presents an example of resource administration, another operating-system service turned into a Racket construct. It displays the essence of the launch-many-worlds function, which is used to run students’ distributed programs (Felleisen et al. 2009) in parallel. The function consumes an arbitrary number of thunks and runs them in parallel until all of them have produced a proper value or one of them has signaled an exception. Since the function itself consumes resources, it uses two custodians: its caller’s—to manage the resources of the given thunks—and a new one—to manage its own resources, mostly threads. If any of these thunks raise an exception, the latter custodian is shut down and all of launch-many-world’s resources are released. For a more sophisticated pattern of killing threads safely, see Flatt and Findler’s work on kill safety (Flatt and Findler 2004).

Finally, Racket also internalizes other aspects of its context. Dating back to the beginning, Racket programs can programmatically link modules (Flatt and Felleisen 1998) and classes (Flatt et al. 1998). In conventional languages, programmers must resort to extra-linguistic tools to abstract over such linguistic constructs; only ML-style languages and some scripting languages make modules and classes programmable, too.