On this page:
The Basic Mechanics
Contracts
Organizing Modules (1)
Unit Tests
Organizing Modules (2)
Data Examples
Organizing Modules (3)
Organizing Modules
Why Not Leave It To Tools
Graceful Failure
8.10.0.1

Modular Programming

Matthias Felleisen

31 Aug 2021

Changed in version 1.5: Tue Feb 22 18:34:22 EST 2022, Added a link to Sid Kasivajhula’s blog post.

Changed in version 1.4: Sat Feb 19 16:00:09 EST 2022, Leif Andersen wondered how to fail gracefully.

Changed in version 1.3: Thu Feb 17 11:34:02 EST 2022, Ryan Culpepper sent in some clarifying edits.

Changed in version 1.2: Fri Feb 11 09:11:46 EST 2022, Michael Ballantyne suggested some links and noticed some typos.

Changed in version 1.1: Tue Jan 25 10:41:19 EST 2022, clarify purpose

Changed in version 1.0: Sun Jan 16 19:06:19 EST 2022, initial release

Racket’s module system is a powerful tool for organizing programs. Its contract system empowers programmers to specify module interfaces with any desired precision. Even the most basic unit testing facilities suffice to set up a sophisticated unit test suite. The combination is awesome.

I have used these three for the past decade to organize the code base for reasonably large semester-course projects (about 3-5Kloc, w/o comments; not that lines is a good measure). Over this time I have worked out basic organization principles for modules, tests, etc. that are rather effective for the intended kind of application program.

The proposed organization works well for the creation of stand-alone (binary) executables and their distribution. It builds on the principles of How to Design Programs, placing examples near data definitions, systematically building tests from the examples, using simple tests to illustrate the workings of a function, and so on. For libraries and frameworks, however, a different organization is called for. In these cases, it is probably best to consider an organization that separates functionality from examples and tests into separate files. Doing so it helps with the management of the size of the download. Also see Why Not Leave It To Tools.

Assumption The reader has some familiarity with Racket’s structs and functions.

The reader who is somewhat familiar with modules, contracts, and tests may wish to focus on those sections whose title contains the word “organization.”

An experienced Racket programmer may wish to read just Organizing Modules.

All readers should consider this write-up as an elaboration of the module chapter in the Racket Style Guide.}

The Basic Mechanics

A Racket module is a collection of definitions and expressions. Syntactically it is an S-expression of the shape (module Id Lang X ...) where X ... is a sequence of definitions, expressions, provides, requires, and more modules. The Lang stands for a language in the Racket universe; Id is the name of the module. A provide specification enumerates the identifiers that are visible outside of the module. A require specification brings identifiers from some other module into scope; its argument is a module path: a quoted file name in the simplest case, a path such as racket/list into a Racket collection, and some more.

"module-example.rkt"

(module module-example racket/base
 
  (provide many)
 
  (require racket/list)
 
  (displayln 'module)
 
  (define (many n)
    (build-list n values)))

"module-example.rkt"

#lang racket/base
 
(provide many)
 
(require racket/list)
 
(displayln 'module)
 
(define (many n)
  (build-list n values))

Figure 1: A plain Racket module

Figure 1 displays two ways of expressing a module. The S-expression on the left strictly obeys the form explained above. The code on the right is a concise and equivalent version of the same module. It differs in three points:
  1. the first line specifies Lang via a #lang line;

  2. the module name is inferred from the name of the code’s file container;

  3. the sequence of Xs is not indented.

All of these beautify modules, and so the remaining sections use this concise form.

A module with a module is a submodule. Figure 2 shows a simplistic example of a submodule that is immediately required into its surrounding—or outermodule. The submodule is named 'sub; the require form uses the path (submod "." sub), which consists of a filepath-like information and the symbolic name of the submodule. The modules in figure 2 are essentially equivalent to those in figure 1.

"sub.rkt"

(module module-example racket
 
  (provide many)
 
  (displayln 'module)
 
  (module sub racket/base
    (provide many)
    (require racket/list)
    (define (many n)
       (build-list n values)))
  (require (submod "." sub)))

"sub.rkt"

#lang racket
 
(provide many)
 
(displayln 'module)
 
(module sub racket/base
  (provide many)
  (require racket/list)
  (define (many n)
    (build-list n values)))
(require (submod "." sub))

Figure 2: A Racket sub-module

A (submod ...) form in a require can also use path to import identifiers from other files. For example, writing (submod "sub.rkt" sub) in a file/module in the same folder as "sub.rkt" imports many.

Now here is the key of the submodule approach:

Requiring a module does not require its submodules and requiring a submodule loads only as much of the surrounding module as required.

Details Racket’s language of modules is far more sophisticated than the this section lets on. If interested, consult the relevant chapter in the Racket Guide.

Contracts

A provide specification is a simplistic way to describe the interface of a module. If every provided identifier precisely describes the exported functionality, it may suffice to enumerate just these identifiers. But this is rarely the case.

The most basic step is to add How to Design Programs’s signatures and purpose statements, comments that describe the inputs and outputs of a function in the author’s (imaginary and unchecked) type system. For example, the module from figure 1 may come with comments such as those:

#lang racket/base
 
(provide
  #; (Natural -> [Listof Natural])
  ;; (many n) creates (list 0 ... n-1)
  many)
 
(require racket/list)
 
(define (many n) (build-list n values))
Comments quickly get out of sync, however. Hence, if it is critical that many is never called on anything but a natural number or if the author of the module wants to enforce the promise that it returns a list, more than a comment is needed.

Racket’s sophisticated contract system provides the mechanism for checking such obligations and promises. A contract is actually just ordinary Racket code. In the simplest case, it is a predicate on values. predicates on functions (as computer scientists know), objects, and so on can’t be just checked like predicates on plain values. For these cases, it helps to use a notation similar to type systems to keep them concise and yet readable.

Here is the simplest contract for many:Many contemporary type system could express some of these kinds of contract, too.
#lang racket/base
 
(require racket/contract)
(require racket/math)
 
(provide
  (contract-out
    [many
      ;; (many n) creates (list 0 ... n-1)
      (-> natural? (listof natural?))]))
 
(require racket/list)
 
(define (many n) (build-list n values))
If curious, you may wish to figure out the actual code that enforces the -> contract. And yes, it’s all done with macros. The -> notation says many is a function. Its first piece, natural?, is a predicate that ensures many is called on natural numbers. Its second piece, (listof natural?), checks that many returns a list of natural numbers. Note the contract notation (->, listof) is brought into scope with the first require and natural? with the second.

The purpose statement of many tells a reader that the function’s result is a list whose length is equal to the given argument. Yes, contracts can express this too:
#lang racket/base
 
(require racket/contract)
(require racket/math)
 
(provide
  (contract-out
    [many
      ;; (many n) creates (list 0 ... n-1)
      (->i ([n natural?])
           (result (n)
                   (and/c (listof natural?)
                          (lambda (l) (= (length l) n)))))]))
 
(require racket/list)
 
(define (many n) (build-list n values))
This contract demands some unpacking. Here we go:
  • ->i tells a reader that the pieces of the contract depend on each other;

  • ([n natural?]) specifies a name and the contract of many’s argument;

  • (result (n) ...) signals that the contract for the result depends on n;

  • (and/c (listof natural?) ...) means the contract of the result consists of several pieces, the first of which is the same as above;

  • (lambda (l) ...) is the second piece of the and/c contract, and it is a predicate that checks whether the list is of length n.

At this point, the contract almost expresses the entire purpose statement. Let’s express some more:
#lang racket/base
 
(require racket/contract)
(require racket/math)
 
(provide
  (contract-out
    [many
      ;; (many n) creates (list 0 ... n-1)
      (->i ([n natural?])
           (_ (n)
              (and/c (listof natural?)
                     (lambda (l) (= (length l) n))
                     (or/c empty?
                           (lambda (l)
                             (and (= (first l) 0)
                                  (= (last l) (- n 1))))))))]))
 
(require racket/list)
 
(define (many n) (build-list n values))
The third piece of the and/c contract checks that the resulting is either empty or satisfies a predicate that checks whether the first item of the list is 0 and the last one is n-1.

If curious, consider writing a contract that completely replaces the purpose statement. Doing so should not be a mystery.

Details Contracts come with a cost. This essay does not address this point. If interested, consult the relevant chapter in the Racket Guide. The essential advice is to articulate contracts that express important properties, usually less than a full specification.

Organizing Modules (1)

When a contract becomes complicated, it is best to define it separately. In the running example, the result contract consists of three pieces, combined via and/c. The first piece is constructed from built-in contract notation; but the second and third are user-designed. Indeed, the third piece alone covers three lines. All of this calls for giving the contract and its pieces meaningful names.

The question is how to organize the resulting code.

#lang racket/base
 
(require racket/contract)
(require racket/math)
 
(define (list-of-nats-from-0-to n) ; exclusive
  (and/c (listof natural?)
         (of-length n)
         (from-zero-to n)))
 
(define ((of-length n) l) (= (length l) n))
 
(define (from-zero-to n) ; exclusive
  (or/c empty?
        (lambda (l)
          (and (= (first l) 0)
               (= (last l) (- n 1))))))
 
(provide
 (contract-out
  [many
   ;; (many n) creates (list 0 ... n-1)
   (->i ([n natural?]) (_ (n) (list-of-nats-from-0-to n)))]))
 
;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(require racket/list)
 
;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(define (many n)
  (build-list n values))

Figure 3: Organizing a module with contracts

Figure 3 displays the result of this refactoring step. It also demonstrates the first important principle of module organization:

All pieces of the interface contract are placed near the top.

Following this advice ensures that others who wish to use the module’s functionality can find its description at a fixed and obvious place, without digging through its implementation. (Yes, the implementation of many is short and sweet, but this isn’t usually the case.)

Let’s take a close look at figure 3:
  • The code above the dashed is the module’s interface.

  • The first require specification imports the contract notation.

  • The second require imports the natural? predicate.

  • The two requires are followed by three contract definitions:
    • The first function, list-of-nats-from-0-to, introduces a descriptive name for the overall contract. It abstracts over the argument, n, and returns the above-described and/c contract.

    • The second piece of the and/c contract also comes with a name: of-length. Its definition introduces a curried function, which consumes the argument n of many and returns the expected predicated on lists.

    • The third piece of and/c is again a function over contracts.

  • The last piece of the contract section is the provide specification. This is preliminary; see Organizing Modules.

Unit Tests

The final step of the design recipes in How to Design Programs calls for the translation of worked examples into unit tests. A test consists of two pieces: input values for the arguments and an expected output value.

Like most contemporary programming languages, Racket comes with a unit test framework, RackUnit. With this testing framework, expressing a unit test is straightforward. Let’s go step by step. We can instantiate the purpose statement of many to get a “worked example,” like this:

applying many to 3 should yield (list 0 1 2).

Turning this into a Rackunit test is now quite simple:

(check-equal? (many 3) (list 0 1 2))

These unit tests are just expressions, so running the module evaluates them and reports all failures, if any.

But really, you want to put these unit tests in a special place. Here is how that works:
#lang racket/base
 
(require racket/contract)
(require racket/math)
 
(provide
 (contract-out
  [many
   ;; (many n) creates (list 0 ... n-1)
   (-> natural? (listof natural?))]))
 
(require racket/list)
 
(define (many n) (build-list n values))
 
(module+ test
  (require rackunit)
 
  (check-equal? (many 3) (list 0 1 2)))
This code snippet introduces (module+ ...) as a submodule form.

The (module+ Id ...) form creates a new submodule with the name Id. Unlike (module Id Lang ...), such a module inherits the language of the surrounding module and all definitions in the surrounding module are implicitly required. That is, these definitions are automatically visible within this submodule.

DrRacket, the Racket IDE, knows to evaluate unit tests in a sumodule automatically. That is, every time a module is run, the IDE also runs the test submodule. If you prefer command-line tools, run

    $ raco test module-example.rkt

at the prompt of your OS shell to evaluate your unit tests in the submodule.


image
Contracts and Tests

Now suppose you’d add the following expression to this kind of submodule:

(many -3)

When you ask DrRacket to run the program now, it will signal an error and the IDE will highlight the (build-list ...) expression.

Why didn’t the contract catch this bad argument value?

While a (module+ ...) submodule auto-imports all definitions, it does so directly without going through the surrounding module’s interface. In the terms of the contract library, the bad argument value does not cross the module boundary between the outer module and the submodule.

If you want to test the contracts of your module via a module+ submodule, there’s an easy way to do so. The following specification

(require (submod ".."))

imports the outer ".." module’s exported identifiers across the module boundary. And if you want to test the contract now, it’s straightforward:

(check-exn exn:fail:contract? (λ () (many -3)))

Details The RackUnit framework is an amazing library. If you are unfamiliar with its capabilities, read the guide to RackUnit.

Organizing Modules (2)

Passing unit tests are reassuring. They certainly expose typo-level bugs. They occasionally point out flaws in the logic. And they serve as a simple form of documentation, which is why having them in the same file is a good idea.

In an untyped language such as Racket, it is also good practice to document calling conventions next to the implementation of a function. Then, when a developer has to study the function definition (rather than just read a module’s interface), a nearby working test can demonstrate the basics of a function—without repeating the signature and purpose statement locally.

Once again, module+ submodules help implement this idea. When the compiler encounters a module, it first creates a single submodule from all the module+ submodules that share the same name. It accomplishes this by concatenating the code of these modules. The synthesized complete submodule is placed at the end of the outer module.

#lang racket/base
 
(require racket/contract)
(require racket/math)
 
(provide
 (contract-out
  [many
   ; (many n) creates (list 0 ... n-1)
   (-> natural? (listof natural?))]))
 
;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(require racket/list)
 
;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(module+ test
  (check-equal? (many 3) (list 0 1 2)))
 
(define (many n)
  (build-list n values))
 
;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(module+ test
  (require rackunit)
  (require (submod ".."))
 
  (check-exn exn:fail:contract? (λ () (many -3)))
  (check-equal? (many 3) (list 0 1 2))
  (check-equal? (many 0) (list)))

Figure 4: Organizing a module with tests and contracts

Figure 4 displays the proposed module organization. It realizes two related guidelines. Here is the first one:

Unit tests go into a module+ submodule at the bottom of the program.

The second one carves out an exception from the first:

Place illustrative unit tests next to function definitions, also in module+.

Following these two pieces of advice has a number of consequences:
  • First, the tests come with the functionality, meaning there’s no need to search the file system for them.

  • Second, a developer looking for the tests navigates to the bottom of the module.

  • Third, simple tests close to a function’s definition illustrate its workings for the reader.

  • Fourth, running the tests is straightforward; see DrRacket and raco above.

  • Finally, even though the tests are placed together with the functionality, a production run of the software system need not require the tests and thus does not load the test code; the code size remains what a developer expects.


image
Running Tests Selectively

Factoring out tests into separate submodules adds another advantage. Say your module includes two classes of functions for some data: regular functionality on data and functions that visualize data. Developing or maintaining this second class of functions is quite different from the first one; as a developer, we want to see what these function produce, not just test their functionality. Then again, these look-and-see expressions also feel like tests.

Where should they go?

Racket places no limit on how many different submodules a module may contain, and its tool chain supports them properly. So, let’s assume that proper tests of visualization functions go into a test module and the development tests go into picts. The first kind of test should run whenever a unit-testing tool is run, and the second kind should run selectively, say, during development in the IDE or selectively from the command line.

Here is how this works:
  • The Language preferences of DrRacket actually allow developers to name the submodules that should be evaluated for each run. Two are set by default: test and main. This is why testing worked so naturally with the examples above.

    To enable the auto-run of picts, a developer merely has to add this name, and everything just works.

  • The raco-test tool comes with an optional -s flag, which allows the developer to tell the unit-testing tool which submodule to run.

    By default, raco test assumes the test submodule is to be tested.

    So, if you also want to run only the tests in the picts submodule from the command line, just specify the name on the command line:

        $ raco test -s picts module-example.rkt

In short, the combination of the simple unit test features and submodules enables a sophisticated management of unit tests.

Data Examples

A non-trivial code base consists of many modules in many files. At least one of them is usually dedicated to a central data structure, which is then referenced in other modules.

To make this a bit more concrete, let us say we are implementing a board game that “automated” players use under the supervision of some referee (a rule enforcer). In this case the central data structure module would encapsulate the game board and the functionality for manipulating it, up to some constraints.

Here is a sketch of such a game-board module:

"game-board.rkt"

#lang racket
 
(provide
 (contract-out
  [initial-game-board  (-> natural? natural? game-board?)]
  [move-piece-on-board (-> game-board? posn? posn? game-board?)]
  [evaluate-a-board    (-> game-board? natural?)]))
 
;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(struct posn [x y])
(struct game-board [state])
 
#; {type Board = (game-baord [Listof [Listof Natural]])}
#; {type Posn = (posn Natural Natural)}
 
(define (initial-game-board x y) ...)
 
(define (move-piece-on-board b from to) ...)
 
(define (evaluate-a-board b) 0)

...

The various names of the structures, data definitions, and functions indicate particular uses, but you will have to use a bit of imagination to make this code sketch truly meaningful.

Following How to Design Programs again, a developer would create sample instance of the data definitions before working through examples and then turning the examples into a test. The question is where to place these data examples. Since they are needed for only the tests, it seems to make sense to place them in the test submodule:

"game-board.rkt"

;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(module+ test
  (require rackunit)
 
  (displayln '[testing the board] (current-error-port))
 
  (define board1 (initial-game-board 3 3))
  (define posn1  (posn 0 0))
  (define posn2  (posn 3 3))
 
  (check-equal? (evaluate-a-board
                 (move-piece-on-board board1 posn1 posn2))
                1))

Back to the context. The players might consist of a module that coordinates the activities of several strategy modules. The referee manages the actions of the various players and checks that a player’s requested action is legal according to the rules of the game. These secondary modules will then import the game-board module. Here is a partial module-dependency graph for this situation (some dependencies omitted to avoid clutter):

image

The game-board is the foundation on top of which all modules are built. That is, the modules at the top of the graph depend on those near the bottom.

A developer will not wish to make instances of the game-board data representation in each of these test modules. Instead, he will use Racket’s provide-and-require mechanism to import data examples from other test modules. Concretely, the developer of "game-board.rkt" equips its test submodule with a provide specification that exports the data examples:

"game-board.rkt"

...

(module+ test
  (provide board1 posn1 posn2))

...

Dually, the developer of, say, the rule-checking module may contain a test submodule that starts like this:

"rule-checker.rkt"

...

(module+ test
  (require rackunit)
  (require (submod "game-board.rkt" test)))

...

(module+ test
 (check- .. board1 .. posn1 .. posn2 ..))

A revised module-dependency diagram clarifies how modules and submodules require each other, respectively:

image

The top three modules require the game-board modules (red), and their respective test submodules require the test submodule from game-board (black). By arranging submodules in this way, all the advantages mentioned in Organizing Modules (2) are preserved, especially the last one of keeping the deployed code size reasonably small.

Sadly this seemingly obvious arrangement does not work out well. Running the tests with either raco (or DrRacket) will show that every require of the test submodule in "game-board.rkt" re-runs its The RackUnit package provides mechanisms for getting around this problem, too, though at the cost of significantly more unit-test code and more complicated code. The solution presented here keeps things simple, easy to read, and easy to run. unit tests, too. Specifically, the displayln line—see highlight in the above test submodule—prints the list as many times as the test submodule is required (plus the original one). This re-execution of tests is both superfluous and misleading in that it reports a lot more tests than you actually have. It must be avoided.

Organizing Modules (3)

So what is the right way to set up a pool of re-usable examples?

You may have already guessed that module+ comes to the rescue again. This time, the solution is rather simple. It calls for setting up an examples submodule and to have the test submodule require the examples submodule.

#lang racket
 
(provide
 (contract-out
  [initial-game-board  (-> natural? natural? game-board?)]
  [move-piece-on-board (-> game-board? posn? posn? game-board?)]
  [evaluate-a-board    (-> game-board? natural?)]))
 
;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(struct posn [x y])
(struct game-board [state])
 
; {type Board = (game-baord [Listof [Listof Natural]])}
; {type Posn  = (posn Natural Natural)}
 
(define (initial-game-board x y) '...)
 
(module+ examples
  (provide board1 posn1 posn2)
 
  (define board1 (initial-game-board 3 3))
  (define posn1  (posn 0 0))
  (define posn2  (posn 3 3)))
 
(define (move-piece-on-board b from to) '...)
 
(define (evaluate-a-board b) 0)
 
;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(module+ test
  (require (submod ".." examples))
  (require rackunit)
 
  (check-equal? (evaluate-a-board
                 (move-piece-on-board board1 posn1 posn2))
                1))

Figure 5: Organizing a module with tests and examples

Figure 5 displays the proposed organization of the game-board module with two submodules: one for examples and one for tests. See the two highlights for the two most crucial lines. The provide line in the examples submodule exports the identifiers of examples. The require line in the test submodule makes these identifiers of data examples available to the suite of unit tests.

This arrangement adds two advantages to those of Organizing Modules (2):
  • It creates a distinct place for data examples, which like unit tests, is easy to locate for a reader of the module’s implementation.

    Other modules can require the data examples into their own examples and test submodules to build up compound examples and add tests.

    For example, a strategy module may wish to represent game states, which consist of game boards plus additional data. Its examples submodules could import the one from the game-board module and build game-state examples using those exported boards.

  • It makes it simple to run all unit tests in a code base with raco-test.

    A developer may wish to use a script that runs raco-test in a particular order on various folders or even several scripts to run only the unit tests of some modules.

Figure 6 displays the resulting dependency graph for the running example: with each test submodule importing each internal examples submodule; with some test submodule importing some external examples submodule; and modules depending on each other, too.

image

Figure 6: The dependency graph with test and example submodules

Organizing Modules

The presented organization is workable, but taking the extra step to collect things that belong together pays off. Roughly speaking, a module consists of up to five sections; on occasion a module may have fewer sections than that, and on some occasions, a module’s functionality code base needs to be broken into several (sub)sections.

#lang racket
 
... module-interface ...
    -- require needed for the module-interface
    -- contracts
 
    -- provide
 
... dependencies ...
    -- require
    -- internal require
 
... data definitions
    -- structure and/or class definitions
    -- data examples
 
... more functionality implementation ...
 
... unit tests ...

Figure 7: Organizing a module into sections

See figure 7 for a sketch. Each of the five sketched sections serves a specific purpose:
  • The module-interface section explains the services that a module and its submodules provide in terms of signatures, purposes statements, and/or contracts. A developer who wishes to look up something about the module’s services should not have to look further than this section.

    This separation works if the names of contract elements directly bring across what the predicates check or if contracts are defined right here in this section.

  • The dependencies section specifies on which other modules this one depends and how its submodules depend on each other. A code reader who wishes to get a quick overview of the relationships in a software system can focus on these sections.

  • Most modules introduce a data representation; some export it. The data representation is specified in the data definitions section. Data definitions are often abstract. To bring across their meaning, it is best to illustrate them with examples, which exist in the examples submodule, a part of this section.

  • The module’s functionality section is the (rest of the) implementation. In a class-oriented code base, most of the functionality will exist in the data-definitions section.

  • The unit-test section is the last one. While some illustrative tests might be embedded in the functionality sections, this is the place to look for the comprehensive test suite, corner cases and all.

#lang racket/base
 
;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
; MODULE INTERFACE
 
(require racket/contract)
(require racket/math)
 
(define ((good-posn? gb) p) '...)
 
(provide
 (contract-out
  [initial-game-board  (-> natural? natural? game-board?)]
  [move-piece-on-board (->i ([gb game-board?]
                             [x (gb) (good-posn? gb)]
                             [y (gb) (good-posn? gb)])
                            (_ game-board?))]
  [evaluate-a-board    (-> game-board? natural?)]))
 
(module+ examples
  (provide board1 posn1 posn2))
 
;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
; DEPENDENCIES
 
(require racket/list)
 
(module+ test
  (require (submod ".." examples))
  (require rackunit))
 
;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
; DATA DEFs.
 
(struct posn [x y])
(struct game-board [state])
 
; {type Board = (game-baord [Listof [Listof Natural]])}
; {type Posn  = (posn Natural Natural)}
 
(define (initial-game-board x y) '...)
 
(module+ examples
  (define board1 (initial-game-board 3 3))
  (define posn1  (posn 0 0))
  (define posn2  (posn 3 3)))
 
;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
; FUNCTIONALITY
 
(define (move-piece-on-board b from to) '...)
 
(define (evaluate-a-board b) 0)
 
;; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
; TESTS
 
(module+ test
  (check-equal? (evaluate-a-board
                 (move-piece-on-board board1 posn1 posn2))
                1))

Figure 8: Organizing a module

Figure 8 shows how this five-sections guideline applies to the module of figure 5. Notice that require shows up twice: once in the module-interface section and once in the dependency section. In this latter section, (module+ examples ..) informs a reader that an examples submodule exists and exports some examples; in a complex system, it would be good to choose more descriptive names for these identifiers and/or add purpose statements to them. The data-definitions section introduces the fragment of the (module+ examples ..) submodule that defines the exported data examples.


image
Practice

In practice, the rules work out well in IDEs, such as DrRacket or VSCode, with support for a birds-eye view, often called a minimap. There, sections can be labeled with “large letters” headings that show up in the minimap. Doing so makes for easy navigation and comprehension, especially since the organization collects all related code pieces in one place.

See figure 9 for a first example. It displays (a DrRacket screenshot of) the key data representation of a game implementation. The minimap contains the expected section labels: export for the module interface; dependencies for the require specifications; and data rep. for the definition of data structures and their interpretation.

The remaining sections aren’t just “implementation” or “functionality.” Instead, you see projection, construction, and all paths. That is, the implementation section is organized into three more pieces, each with its own heading.

The test section is at the bottom of the module, as expected.

Figure 9: A sample module from a non-trivial code base

The screenshot in figure 10 shows a module that combines two data representations (from the same code base as figure 9) and some functionality on each. They are in one module because the two are always used together, but each domain of information is sufficiently complex to deserve its own data representation. Each section comes with a distinct header—-I, player and all players–and with a separate test section.

(The second data representation integrates the first, and its tests rely on examples and tests from the first one.)

Figure 10: A module that combines two data representations

Finally, the last practical example illustrates how a module consisting of just functionality may also call for separate sections that include their own test submodules. Concretely, figure 11 is a module that exports one large and complicated piece of functionality. Following How to Design Programs, the function is a composition of several other functions, some of which are composite functions again. Each major auxiliary function gets its own section with its own test submodule. The full “integration” tests for the main function are found at the bottom of the module.

While there is no data def. section, there is a section—near the top—for additional data examples that are needed to formulate unit tests for the auxiliary functions.

Note how the organization is close to how the module is created. Planning quickly shows the need for auxiliaries. So the main function is sketched to ensure the plan could work, and then the sections are created sequentially, each with its own tests. In turn, this process demonstrates the need for additional data examples, which brings about the just mentioned section.

By the time the full tests section is developed, there is a lot of confidence in the workings of the auxiliary functions. (The integration tests for the main function still uncovered some additional bugs in the latter.)

Figure 11: A module with several implementation & test sections

Why Not Leave It To Tools

Many Racket programmers don’t use this strict organization schema. Some prefer to specify all required modules at the top. Others wish to put the provide specifications next to the defined identifiers. See Sid Kasivajhula’s blog post on how to mimic this library-tests-documentation approach of the Racket main distribution. I consider it highly appropriate for package developers. Yet more separate tests completely from functionality, placing them into different files in different collects. This separation is partly due to history and partly to a desire to reduce the size of distributed source code. Indeed, many of these habits date back to a time when Racket was a Scheme and when Scheme was a Lisp.

Especially Schemers have pushed the idea that looking at a file should be done with an appropriate suite of tools. In this day and age, we would say “IDE.” So, if the reader of a module wishes to see just the interface, some “interface extraction tool” would show it. In the same fashion, another tool would gather the dependencies into a single place. Conversely, someone who studies the implementation section could mask out all other material. You can easily imagine a suite of additional tools.

One day all IDEs will rely on language servers and a uniform suite of tools. At that point, developers may no longer have to put in much energy into organizing their modules.

I have been given advice since the late 1980s. To this day, however, no such suitable tool suite exists. Part of the problem is that different developers have different IDE preferences. While different IDEs present code in different ways—see figure 12 for example—the organization of text comes through.

For now, it is best if developers—human beings that they are—accommodate each other with a bit of extra work going into code organization. The approach presented here has worked well for me and people who had to read my code.

Figure 12: The module from figure 10 in VSCode

Graceful Failure

While the organization makes it easy to find the various pieces, it also relies on a strong working discipline. Slipping on occasion or violating the discipline is acceptable. Here are some sample situations:
  • When I am writing a test for the comprehensive test suite at the bottom of a module and the available examples are not what I need. I might just add it to the test module right where I am working.

  • If the construction of an example demands yet another example from another dependency (module), I may open the dependency, find out the example exists, and simply provide it where it is named. Before doing so, I may give it a name that is consistent with the naming discipline for exported examples.

  • Similarly, the discovery that a hidden function is needed may just call for adding a provide next to it for the time being. If the provide of the module uses contracts, the local provide should do so, too; otherwise a signature, formulated as a comment, is enough.

Unless such exceptions pile up, it is usually easy to restore order later.