3 Racket is a Programming-Language Programming Language

Racket is a programming language. Actually, at first glance it looks like a family of conventional languages, including a small untyped, mostly-functional by-value language (racket/base), a batteries-included extension (racket), and a typed variant (typed/racket).

demo.rkt

#lang racket

(provide

; type Video = [Listof Image]

; Natural -> Video

walk-simplex)

; IMPLEMENTATION

(require "small.sim" 2htdp/image)

; Natural -> Video

(define (walk-simplex timing)

... (maximizer #:x 2) ...)

Figure 1: A plain Racket module

Like all programming languages, plain Racket forces the programmer to formulate solutions to problems in terms of its built-in programming constructs. But, Racket is also a member of the Lisp family, which has always insisted on stating solutions in the most appropriate language, one suited to the problem domain. As Hudak (Hudak 1998) puts it, “domain-specific languages are the ultimate abstractions.”

Following this reasoning, each program component is articulated in the Racket-based programming language that is best suited for the problem it solves. If the language is not available, the Racket programmer creates it, possibly even for a single module. To support this kind of system building, Racket is a programming-language programming language.

small.sim

#lang at-exp simplex

; provides: synthesized function maximizer:

;    #:x Real -> Real

;    #:y Real -> Real

#:variables x y

3 * x + 5 * y <= 10

3 * x - 5 * y <= 20

Figure 2: A module for describing a simplex shape

Figure 1 and Figure 2 illustrate the principle. The figure specifies the filename. It is not a part of the code. The astute reader will notice that this violates the principle of keeping everything in the language. The first module in figure 1 uses the racket language, which is specified in the so-called #lang—pronounced “hash lang”—line. The module provides a single function; the comments inside the provide specification informally state a type definition and a function signature in terms of this type definition. To implement this function, the module uses (require "small.sim") to import functionality from the module in figure 2 and then defines its own functions.

The creator of the module in figure 2 prefers a domain-specific language, because the module’s purpose is to synthesize a function for a simplex, and the most natural way to specify the latter is to state a collection of linear inequalities. The comments below the #lang line in figure 2 state that the module exports a single function, maximizer. Concretely, the #:variables specification and the following inequalities determine the maximizer function. When called as (maximizer #:x n), the function produces the maximal y value; conversely, (maximizer #:y m) delivers the maximal x value.

In support of this kind of language-oriented programming, Racket provides a syntax extension system that borrows elements from Scheme’s macro system (Dybvig et al. 1992; Kohlbecker et al. 1986; Kohlbecker and Wand 1987) but also improves on it in several different directions. First, the Racket syntax extension system is about defining languages (Flatt 2002; Krishnamurthi 2001; Krishnamurthi et al. 1999), not just extending an existing language with new linguistic constructs. For example, Racket’s class system (Flatt et al. 2006), its first-class components (Flatt and Felleisen 1998), and its language of (loop) comprehensions are just such sub-languages, though their constructs are indistinguishable from Racket’s core features. Naturally, a Racket-based language is just a module whose exports make up a new language. These exports must include certain features and may otherwise come with any syntactic constructs and run-time values deemed necessary. The module may define these exports or may import and re-export them from an existing language. Hence, a language module can easily add features to, or subtract them from, an existing language.

This example is inspired by the teaching languages (Felleisen et al. 2001). In particular, the first-order functional teaching language uses it to check whether the function position is a name defined by the program or the language so that it can produce novice-friendly error messages when something else shows up. However, the pattern is used much more widely. For instance, the FrTime language uses this same mechanism to create a dataflow variant of call-by-value (Cooper and Krishnamurthi 2006).

Third, Racket’s syntax extension system grants a language-defining module access to the entire syntax tree for a guest module, not just individual nodes in the syntax tree. This access allows the collaboration between the rewriting rule for #%app and define in the above example. Indeed, this kind of communication smoothly generalizes to complex context-sensitive analysis tasks and, in particular, allows for the implementation of a rather conventional type checker (Tobin-Hochstadt et al. 2011).

Fourth, Racket comes with a library that supports the programmatic creation of lexers and parsers (Owens et al. 2004). It is thus possible for a language implementation to transform conventional syntax into regular S-expression syntax and to subject this result to the conventional syntax extension system and its rewriting rules. See figure 2 where the implementor of a domain-specific language prefers an ASCII-mathematics notation. Importantly, the separation of parsing from the syntax extension naturally creates an interface between unrelated parts of language design—notation and meaning—and thus enables language engineers to factor the work into two independent components: design of surface notation and meaning.

Finally, Racket insists on separating the various stages of language processing, particularly enforcing a strict separation of compile-time from run-time code. For example, the rewriting rules generate pure syntax and may not embed other language values inside this syntax. Similarly, since the world of Racket languages is actually an inverted pyramid of languages atop languages, each language-processing module may have side-effects—and these side-effects must be insulated from the rest of the language-processing pipeline.

In sum, Racket’s toolbox empowers programmers to create new languages quickly and thus enables language-oriented program design. The key to this achievement is to improve over Lisp and Scheme’s approaches: Racket carefully stages syntax elaboration (Flatt 2002), eliminating Lisp’s problematic eval-when-where approach; it enables the quick derivation of new languages from existing ones; and it enables the introduction of conventional syntax.