[dev] The Apollo Display Manager [very long] from John Yates on 2011-06-24 (dev mail list archive)

From: John Yates <john_AT_yates-sheets.org>
Date: Fri, 24 Jun 2011 14:13:30 -0400

While following the st and editor threads I felt prompted to put down
these reminiscences. My hope is that they will stimulate the
discussion.

/john

INTRODUCTION

Many years ago I worked at Apollo Computer.[APOLLO] This was before
X, NFS, Sun workstations or Apple Macintoshes; when Unix typically
meant time sharing a VAX via a glass TTY; when so many features of
today's computing landscape (vi, emacs, x86 PCs, etc) were in their
infancy; when GUIs, C++, and numerous other forms of software bloat
were still in the future.

Into this world Apollo offered the first commercial 3M machine.[3M]
An Apollo "node" provided demand paging, network-wide virtual memory,
windowed bit mapped graphics and a user interface that encouraged
concurrent activities.[ARCH, DOMAIN] Given one MIP of computes and at
most 1MB of RAM delivering an experience compelling enough to justify
a price well in excess of $10K was no easy task. Not surprisingly the
suckless virtues of simplicity, clarity, and frugality permeated both
the design and the codebase.

One of the most beloved elements of the Apollo user experience was the
Display Manager (hereafter simply the DM). It combined the roles of
window manager, key stroke mapper, script interpreter, text editor and
virtual terminal in a very intuitive and satisfying manner.[DM]

DESCRIPTION

= Windows and pads =

The DM rendered the contents of objects called "pads" within pannable,
re-sizable windows.[GS, p 1-11] A pad was a higher level abstraction
than a simple graphics canvas. It was a sequence of 1D lines of text
and 2D frames. A frame in turn could contain either graphics or
text. A pad could be anonymous or it could be backed by a named file.

= Universal text =

As a text canvas a pad was the obvious building block for any feature
requiring rendered text. The DM provided a very standard set of text
editor operations on pads. Non-destructive operations (navigation,
search, mark, copy, etc.) were universally available. Destructive
operations (insert, delete, cut, paste, replace, undo, etc.) were
disallowed in read-only pads.

= Viewing and editing text files =

One could create a pad, bind it to a file and seed it with that file's
contents. If the pad was created as read-write then one had a classic
text editor; if read-only then one had a text file viewer. A modified
pad could (but did not need to) be written back to disk.

= Pads attached to streams =

Pads really shown when they were connected to process I/O streams.

A pad sinking an output stream was termed a "transcript pad". It
maintained a complete, scrollable, searchable log of the data written
to that stream. Though typically anonymous at creation a transcript
pad could subsequently be assigned a file path, allowing it to persist
after all connected processes exited. By design, transcript pads were
permanently read-only, ensuring that recorded history remained
immutable.[EMACS]

A pad sourcing an input stream was termed an "input pad". It very
naturally provided editable type-ahead. An input pad was always
associated with a transcript pad.

The DM provided an extra bit of integration around an input pad /
transcript pad pair. The pair were rendered as separate panes within
a single window: transcript above, input below, separated by a thin
horizontal line. Typically a shell or interactive process was created
with its stdin connected to an input pad and its stdout connected to
the associated transcript pad. This arrangement was often called a
"virtual terminal".[TERM]

The DM even understood the idiom of command line prompts. A partial
line (no trailing '\n' ) written to stdout followed by a read request
on stdin was treated as a prompt when there was no complete input line
(terminated by '\n') available or when the input pad was on hold. In
this case the prompt text was inserted at the left topmost position of
the input pad. Once a completed input line became available it got
forwarded to the waiting process. Simultaneously both the prompt and
the associated input were transferred from the top of input pad to the
bottom of the transcript pad.

= Single name space for commands =

All DM functionality (window management, pad management, text input,
navigation, editing, etc.) was invoked via named commands invoked
through a single interface. For this reason working with the DM felt
immensely consistent and integrated. It did not matter whether one
was moving or resizing a window, creating a shell in a virtual
terminal, viewing a file, requesting re-execution of the most recent
shell command, searching an expanse of text, moving focus to another
window, entering text, or any other operation. All available commands
lived in a single name space [REF, pp 1-13:1-17]. The existence of
this single vocabulary of commands made having a single execution
engine (interpretter) completely straight forward and intuitive.

= Command execution =

Syntactically a command invocation consisted of a prefix geometry
specifier, a short identifier naming the desired command, a fixed
number (possibly zero) of arguments and optional flags to modify the
command's detailed behavior.[UG, pp 3-4:3-10] Missing geometry
components most typically were taken from the most recent mark or from
current mouse/cursor position. The limiting case in which all
necessary geometry was missing devolved to using the current value of
the mouse/cursor, making operating at the mouse/cursor position simply
a special case of a more general mechanism. Command arguments could be
expressed as in-line literals or they could be fetched during command
execution in response to a DM prompt.

Commands were excuted via the DM command interpreter. The visible
path into the interpreter was via the DM command window at the bottom
of the screen.[MINI] This window was a horizontal variant of the
classic input/transcript pair. The input pane displayed prompts and
provided a standard input editing environment. The transcript pane
displayed DM output and diagnostics.

= Scripting interface =

The DM interpreter was excruciatingly limited. It provided no
variables, no expressions and no conditionals. The only form of
control flow was sequential execution. Though rarely needed the
script writer's escape hatch was the ability to output to and then
interpret an external file.

= Soft keyboard =

The DM keyboard was entirely soft, defined by a user supplied
initialization script.[UG, pp 3-18:3-21] At its core the key binding
mechanism was modeless. The value of a key binding was simply a
string to be submitted to the interpreter when that key was pressed.
Because a single key stroke could rebind multiple keys users were able
to implement elaborately modal schemes.[MODAL] It was striking how
responsive such schemes were in practice. I suspect that this was
because the binding operation simply recorded strings without
performing any validity checks; checking occurred when and if a new
binding ever got executed.

= Coordinate systems =

Many commands needed a locus at which to operate. Typically window
operations worked with screen coordinates while pad operations worked
with line number and character offset. Either form of locus could be
expressed as a command's geometry specification. There was one
geometry syntax to convey an absolute pixel position. There were two
geometry syntaxes to convey a pad text position: one specified a line
number and character offset directly, the other specified them
indirectly via the result of a forward or backward regular expression
search. More importantly, all forms of geometry specification were
equivalent. The DM could always translate in either direction based
on the needs of the current command.

THOUGHTS AND OBSERVATIONS

= Binding keys at compile time triggered the modal / modeless controversy =

As I wrote up these memories it occurred to me that at Apollo we never
had arguments about key bindings. Bindings were deemed entirely
malleable and matters of personal taste. Our creativity went into
defining useful DM commands and into composing scripts using those
commands to accomplish various tasks. Once a script existed each user
was free to invoke it via whatever binding (s)he might choose. From
this perspective the suckless proclivity for freezing key bindings at
compile time is overly constraining.

= Virtual terminal versus emulation of dumb terminals =

The Apollo team was fully aware of the technique of terminal
emulation. In fact the system call reference manual mentioned dumb
terminal emulation as an appropriate application of pad frames.[SC, p
5-42] That the *nix world offers so many terminal emulator variants
suggests to me that the basic concept is deficient.

The DM was written at a time when many of us still remembered working
on hard copy terminals (DECwriters and actual TeleTypes). The
immutability of hard copy was a given. We very much valued the
ability to review entire sessions, even after disconnecting.
Transcript pads obviously were an attempt to recover that experience.

Input pads were not so much an attempt to recover something lost as an
attempt to resolve the eternal tension between echoing keystrokes and
providing feedback during type ahead. Over the years I have seen this
addressed in many ways: in keyboard drivers, in terminal emulators, in
readline-style libraries, etc. Typically one accepts either the
co-mingling of characters typed ahead and program output or that type
ahead remains hidden unless one uses various magic key bindings. DM
input pads solved this problem very nicely by echoing type ahead
immediately in the input pad and then redisplaying it in the
transcript at the point that it actually got consumed. Further,
having the DM assume responsibility for echoing simplified the
keyboard driver to the point that did nothing more than buffered and
forwarded key strokes.

= Editing type ahead =

Quite apart from echoing type ahead while the previous command is
generating output there is the separate issue of editing incomplete
command lines. Every input stack I have ever used seems to develop at
least some functionality in this realm, even if it is simply back
spacing to delete the most recently typed character. More elaborated
input "cooking" mechanisms attempt to harmonize with well-known
editors by mimicking their default key bindings. Of course that
harmony fails once one uses a less well known editor or customizes
one's key bindings.

DM input pads finessed this problem. No Apollo readline-like
capability ever existed. Apollo commands read directly from stdin and
consumed the result without further processing. Users were forever
free to redefine their key bindings to suit their whims. Editing
one's command input always honored one's idiosyncratic key bindings.

= Fewer mechanisms meant easier mastery =

A reaction I often heard from DM users who had had to move to Unix was
"Yuk! What at cognitive burden! Why are there so many distinct
mechanisms, contexts, modes, syntaxes, and vocabularies of key
strokes?"

As a reification of history the DM transcript was an immediately
grasped yet powerful enabler. Take for instance the task of renaming
all files in a directory whose names match a given pattern to related
names...

A *nix shell guru might know enough shell syntax and have the
confidence to perform this as a one line for loop. A mere mortal
would execute "ls <pattern>", redirect the result to a temporary file,
open that file in an editor, use a regular expression to turn the
listing into a set of rename commands, save the file, chmod the file
to be executable, and finally execute the file at the shell prompt.
(Those slightly more knowledgeable might avoid the chmod step by
sourcing the file.)

By contrast most DM users had a key bound to a
copy-last-command-output-to-input-pad script. In detail this script
implemented the sequence:
- put the input pad on hold
- switch to the transcript pane
- goto the bottom
- place a mark
- search backwards for a shell prompt
- move forward one line
- copy the region between point and mark to a paste buffer
- switch to the input pane
- goto the bottom
- place a mark
- insert the contents of the paste buffer
- goto mark

With access to this script it should be obvious how one proceeded:
execute "ls -1 <pattern>" (no need for redirection), hit whatever key
invoked copy-last-command-output-to-input-pad, use a regular
expression to turn the listing into a set of rename commands, release
hold. Fewer steps, no naming a temporary file, fewer key strokes.
And as an added benefit the complete sequence of renames was recorded
in the transcript, not buried in some ephemeral file.

Or take the ubiquitous *nix shell history mechanism. The Apollo shell
had no analogous mechanism. Instead there was a
copy-previous-command-to-input-pad. It to operated by searching
backward through the transcript. In contrast to a *nix shell which
would simply seeds the input line with an earlier command this DM
script would do all of that but would also reposition the transcript
within the window so as to display that duplicated command in its
context along with its output.

Yet another example was view-filename-at-cursor. As originally
shipped this script simply used regular expression searches to find
the two ends of a filename and to submit it as an argument to CV
(create-view). Effectively this was a request to open a file in one's
current working directory. The script was not very useful once one
had changed to a different working directory. In time a variant
script emerged that exploited the fact that shell prompts displayed
the working directory. This version picked up the filename, searched
backwards for a shell prompt, extracted the embedded directory path,
and finally submitted to CV the concatenation of the directory path
and the filename.

REFERENCES AND FOOTNOTES

[3M] http://en.wikipedia.org/wiki/3M_computer

[APOLLO] http://en.wikipedia.org/wiki/Apollo_Computer

[ARCH] http://www.bitsavers.org/pdf/apollo/Apollo_DOMAIN_Architecture_Feb81.pdf

[DM] http://en.wikipedia.org/wiki/DM_(computing)

[DOMAIN] http://en.wikipedia.org/wiki/Apollo/Domain

[EMACS] Contrast this with the fragility of the transcript emacs
creates when running a shell in a buffer.

[GS] http://www.bitsavers.org/pdf/apollo/002348-01_Getting_Started_With_Your_Domain_System_Oct83.pdf

[MINI] In emacs terms this would be analogous to a single dedicate
mini-buffer frame servicing all windows on the screen.

[MODAL] http://groups.google.com/group/comp.sys.apollo/msg/d0294a7db6e8bb0a

[REF] http://www.bitsavers.org/pdf/apollo/002547-04_DOMAIN_System_Command_Reference_Jun87.pdf

[SC] http://www.bitsavers.org/pdf/apollo/008858-00_Programming_With_General_System_Calls_Mar86.pdf