Lecture 2: Software Systems

So what is a software system?

What is a software system?

For our purposes, a piece of software is dubbed a system if it has components that

• deal with a start-up phase,

• maintain a steady-state phase,

• shut it down gracefully on failure, and

• manage its own configuration.

A system consists of several software components, with clearly delineated responsibilities with interfaces and protocols that describe their interactions. Example: Your OOD project (game, music editor) came with a model, a view, and a controller.

A system may also consist of distributed components. In this course we will construct such a system in a systematic manner from a monolithic system. Example: Your phone’s voice-command system has a local component but often needs to check in with a server to synthesize answers.

A system is configured during start-up. Example: Many of you may have shell-configuration files, especially if you’re on *nix or Mac.

A system may also allow re-configuration at run time. Example: When you launch an application on your computer aka phone, it plugs into the underlying operating system for services. Conversely, your operating system is extended with a new component.

Examples: operating systems, web browsers, web servers, and many other well-known pieces of software are such systems. Our system will allow dynamic extensions for configuration purposes and perhaps passive ones while it runs.

Large systems consist of systems, but a semester is too short to construct meaningful examples of this kind.

How do you develop a software system?

A long long time ago, one of the stone-age software engineering researchers was asked to describe a software construction process. He put up a strawman that generalized the above idea of how software comes about into a so-called waterfall model. (His name is Barry Boehm.)

Modern software developers babble "agile" and "sprints" and "tdd" and other such words. Some of the suggested readings will make you buzzword compliant in this regard.

See above for what university-level courses can and cannot do. If all you want is vocational sw dev knowledge, a bootcamp or a trade school will serve you better.

Every company that employs you will add its own twists to this aspect.

What I know is this:

• someone needs to know where you want to end up

• this someone must identify essential components

• and must know how to arrange the construction of these pieces so that you can demo asap

What this means is that someone creates an overview picture, figures out (crude) dependencies, and identifies runnable milestones. This is called top-down planning because you start from the big picture and go down to details. Then it’s time to make a plan to build these pieces like a solid house:

• foundation first

• load-carrying walls

• .. all the way to the roof.

That’s called bottom-up coding.

A Short Detour

This idea doesn’t just work in software but in many fields. Here is one concrete example: the construction of a house.

Imagine you want to build a house, not just buy some cookie-cutter ’thing’ from a (home) developer or an old 1918 condo.

Top-down does not mean “start with the roof, end up with the foundation.” It is really a reference to the idea of a holistic vision about the house you want.

So, you will collect ideas: on the facade, the roof line, the front porch, the placement of the house, the style elements (for example, the tiles you embed in the stucco), and so on. You will also think about the interior: the number of bedrooms, where they are located; whether you want a formal dining room; whether it’s a play-room den and/ora formal living room. You also want to figure out the relative placement of these rooms, and perhaps some more details. Then you will meet with an architect, who will listen, interpret, suggest changes, and make some quick sketches.

At that point, (you think) you’re done but the work begins. The architect must translate this into various forms of blueprints and pretty drawings. The drawing are for you; the blueprints and plans are for the contractor and the subs.

You review and you correct and the architect changes the plans. This feedback loop goes on for a while, and then you agree to start building.

Bottom-down does mean that you start with the construction of the foundation but a lot of prep work goes into a construction before you start and the very act of construction will modify the top-down planning.

Software Construction Models

Let’s compare this process with software development ideas that were/are (supposedly) en vogue.

Waterfall

A long long time ago, in a galaxy far away, an early software engineering researcher was asked to adapt this construction process to the world of software. He came up with the waterfall model:

For each stage, a specialized team would pick up where the previous one left off (or where the ultimate customer gets involved) and work out the next intermediate product. When it was done, it would throw its product over the wall and move on to the next project.

Another claim, usually associated with the waterfall model, concerned the cost of bugs. Boehm suggested that a delay by one stage would increase the cost of finding and fixing the flaw by an order of magnitude.

Back then, nobody really believed that anybody could throw a product at any stage over the wall and have the next team move on—-without ever going back and modifying intermediate results from early stages. And nobody had real data to justify the order-of-magnitude claims about bugs. It was just a proposal ("theory") so that people could discuss models of software development. But people often talked about this model of software development as if it were a basic truth.

The Leprechauns of Software Engineering

The key is that all of these steps are actually involved in software development _but_ it is better to imagine them arranged in a spiral than in a linear, "waterfall" fashion. Also, it is critical to eliminate the "walls" that separate the different people who work on the various aspects of the process.

Agile et alii

About 10-20 years ago, people pushed back with development models and propaganda of their own. But these are basically variants of the above:

• test-driven development

• agile development

• extreme programming

The emphasis has definitely shifted from carefully separating these stages to programming and coding and testing. Over time, computers have turned into elements of a permanently networked world-wide system of interconnected software. Hence deploying software is no longer the big _technical_ problem that it once was. We can push a button, and gmail changes on a world-wide basis in almost no time.

Reflect Did the _social_ and _psychological_ barrier to software deployment get reduced to the same extent?

Despite this shift, software development still has all the above elements to some degree. Someone needs to figure out what kind of software to build. Someone needs to translate this goal into specifications for components. Someone will need to code the specifications. Etc. It is just that the actual responsibility shifted.

And yes, it is now understood that we don’t separate coding from testing and specification from testing as strictly as people imagined in the stone age of software development. We understand, for example, that working thru examples illuminates the specifications and prepares the coding. We understand that deployment is part of the specification—people don’t want sudden shifts in interfaces (-> gmail).

True story

My manager once asked me to sort the results of a query that utility workers could make with our software. So we prototyped the sorting algorithm, making sure it fit into 2K and ran reasonably fast. And my manage proudly demoed the new component to the customer .. only to hear him say, "but I want to sort." Hmph .. It turned out the customer had meant "search" and we were lucky we had discovered this problem via a quick demo, rather than after deploying it (and then fixing the bug).

Homework B: Show and Tell

1. Explain the homework problem.

   Command-line arguments and STDIN files are typical, though somewhat simple-minded ways of configuring a software system.

   Also, consuming streams of inputs and producing streams of outputs, plus connecting such streams thru pipes, is also a well-known arrangement of software components. PL researchers have turned this idea into internal language constructs: see Haskell and now Java. Good Software Engineering courses explain such large-scale architectures.

2. Explain Racket solution. We can’t show a solution in a majority language. And I don’t expect many Racket students.