@italic{This software is under development. Set expectations accordingly.}
@section{Installing Quad}
At the command line:
@ -31,9 +34,30 @@ A document processor that outputs to PDF.
A demo language built on Quad that takes a text-based source file as input, calculates the typesetting and layout, and then outputs a PDF.
@section{What can I do with this demo?}
You can fiddle with it & then submit issues and feature requests at the @link["http://github.com/mbutterick/quad"]{Quad repo}. After a few dead ends, I think I'm headed in the right direction. But I also want to open it to comments & criticism, because it can bend the thinking in productive ways.
@section{What are your plans for Quad?}
Some things I personally plan to use Quad for:
@itemlist[#:style 'ordered
@item{@bold{A simple word processor}. Quadwriter is the demo of this.}
@item{@bold{Font sample documents}. In my work as a @link["https://mbtype.com"]{type designer}, I have to put together PDFs of fonts. To date, I have done them by hand, but I would like to just write programs to generate them.}
@item{@bold{Racket documentation}. The PDFs of Racket documentation are currently generated by LaTeX. I would like to make Quad good enough to handle them.}
@item{@bold{Book publishing}. My wife is a lawyer and wants to publish a book about a certain area of the law that involves a zillion fiddly charts. If I had to do it by hand, it would take months. But with a Quad program, it could be easy.}
]
@section{What's a document processor?}
A document processor is a rule-driven typesetter. It takes a text-based source file as input and converts it into a page layout.
A @deftech{document processor} is a rule-driven typesetter. It takes a text-based source file as input and converts it into a page layout.
For instance, LaTeX is a document processor. So are web browsers. Quad borrows from both traditions —it's an attempt to modernize the good ideas in LaTeX, and generalize the good ideas in web browsers.
@ -70,9 +94,9 @@ But web browsers have some limitations:
@item{Web browsers only render HTML, and many typesetting concepts (e.g., footnotes) don't correspond to any HTML entity. So there is a narrowing of possiblities.}
@item{Browsers are built for speed, so high-quality typesetting (e.g., the Knuth–Plass linebreaking algorithm) is off the table}
@item{Browsers are built for speed, so high-quality typesetting is off the table.}
@item{Browsers are inconsistent in how they render pages.}
@item{Browsers render pages inconsistently.}
@item{Taking off my typography-snob tiara here — browsers are unstable. What seems well supported today can be broken or removed tomorrow. So browsers can't be a part of a dependable publishing workflow that yields reproducible results.}
]
@ -94,6 +118,8 @@ While there's no reason Quad couldn't produce an HTML layout, that's an easier p
Much of the font-parsing and PDF-rendering code in Quad is adapted from @link["http://github.com/foliojs/"]{FolioJS} by Devon Govett. I thank Mr. Govett for figuring out a lot of details that would've made me squeal in agony.
For the most part, neither Quad nor Quadwriter rely much on @racketmodname[racket/draw], and completely avoid its PDF-drawing functions. These facilities are provided by Pango, which has some major deficiencies in the kind of PDFs it produces (for instance, it doesn't support hyperlinks). In fact, most of the good ideas in Quad, I figured out a while ago. But then I had to embark on a multi-year yak-shave to be able to get sufficient control of PDFs.
@section{What doesn't Quad do?}
@itemlist[#:style 'ordered
@ -114,13 +140,13 @@ More specifically:
@itemlist[#:style 'ordered
@item{Quad starts with a source file. In this demo, we can will use the @code{#lang quadwriter} language. For the most part, it's text with markup codes (though it may also include things like diagrams and images).}
@item{Each markup entity is called a @defterm{quad}. A quad roughly corresponds to a box. ``Roughly'' because quads can have zero or negative dimension. Also, at the input stage, the contents of some quads may end up being spread across multiple non-overlapping boxes (e.g., a quad containing a word might be hyphenated to appear on two lines). The more precise description of a quad is therefore ``contiguous formatting region''. Quads can be recursively nested inside other quads, thus the input file is tree-shaped.}
@item{Each markup entity is called a @deftech{quad}. A quad roughly corresponds to a box. ``Roughly'' because quads can have zero or negative dimension. Also, at the input stage, the contents of some quads may end up being spread across multiple non-overlapping boxes (e.g., a quad containing a word might be hyphenated to appear on two lines). The more precise description of a quad is therefore ``contiguous formatting region''. Quads can be recursively nested inside other quads, thus the input file is tree-shaped.}
@item{This tree-shaped input file is flattened into a list of @defterm{atomic} quads. ``Atomic'' in the sense that these are the smallest items the typesetter can manipulate. (For instance, the word @italic{bar} would become three one-character quads. An image or other indivisible box would remain as is.) During the flattening, tags from higher in the tree are propagated downward by copying them into the atomic quads. The result is that all the information needed to typeset an atomic quad is contained within the quad itself.
@item{This tree-shaped input file is flattened into a list of @deftech{atomic} quads. ``Atomic'' in the sense that these are the smallest items the typesetter can manipulate. (For instance, the word @italic{bar} would become three one-character quads. An image or other indivisible box would remain as is.) During the flattening, tags from higher in the tree are propagated downward by copying them into the atomic quads. The result is that all the information needed to typeset an atomic quad is contained within the quad itself.
@margin-note{The input is flattened because typesetting operations are easier to reason about as a linear sequence of instructions (i.e., an imperative model). To see why, consider how you'd handle a page break within a tree model. No matter how deep you were in your typesetting tree, you'd have to jump back to the top level to handle your page break (because it affects the positioning of all subsequent items). Then you'd have to jump back to where you were, deep in the tree. That's unnatural.}}
@item{Atomic quads are composed into lines using one of two algorithms. (Each line is just another quad, of a certain width, that contains these atomic quads.) The first-fit algorithm puts as many quads onto a line as it can before moving on to the next. The best-fit algorithm minimizes the total looseness of all the lines in a paragraph (also known as the @defterm{Knuth–Plass linebreaking algorithm} developed for TeX). Best fit is slower, of course.}
@item{Atomic quads are composed into lines using one of two algorithms. (Each line is just another quad, of a certain width, that contains these atomic quads.) The first-fit algorithm puts as many quads onto a line as it can before moving on to the next. The best-fit algorithm minimizes the total looseness of all the lines in a paragraph (also known as the @link["http://defoe.sourceforge.net/folio/knuth-plass.html"]{Knuth–Plass linebreaking algorithm} developed for TeX). Best fit is slower, of course.}
@item{Once the lines are broken, extra space is distributed within each line according to whether the line should appear centered, left-aligned, justified, etc. The result is a list of quads that fills the full column width.}
@ -140,7 +166,7 @@ Open DrRacket (or the editor you prefer) and start a new document with @code{#la
@fileblock["test.rkt"
@codeblock|{
#lang quadwriter
#lang quadwriter/markup
Brennan and Dale like fancy sauce.
}|
]
@ -170,16 +196,18 @@ Next, on the REPL enter this:
> doc
}
You will see the actual input to Quad, which is called a @defterm{Q-expression}:
You will see the actual input to Quadwriter, which is called a @deftech{Q-expression}:
@repl-output{
'(q "\n" "Brennan and Dale like fancy sauce.")
'(q ((line-height "17")) (q ((break "paragraph"))) "Brennan and Dale like fancy sauce." (q ((break "paragraph"))))
}
A Q-expression is an @tech{X-expression} with some extra restrictions.
In the demos that follow, the input language will change slightly. But the PDF will be rendered the same way (by running the source file) and you can always look at @racket[doc].
@subsection{Soft rock: Quadwriter + Markdown}
@subsection{Soft rock: Quadwriter & Markdown}
I @link["https://docs.racket-lang.org/pollen/second-tutorial.html#%28part._the-case-against-markdown%29"]{don't recommend} that writers adopt Markdown for serious projects. But for goofing around, why not.
@ -212,9 +240,11 @@ And they love to code
}|
]
You are welcome to paste in bigger Markdown files that you have laying around and see what happens. As a demo language, I'm sure there are tortured combinations of Markdown notation that will be confusing to @racket[quadwriter/markdown]. But I've tried with big vanilla files, and it's fine.
You are welcome to paste in bigger Markdown files that you have laying around and see what happens. As a demo language, I'm sure there are tortured agglomerations of Markdown notation that will confuse to @racket[quadwriter/markdown]. But with vanilla files, even big ones, it should be fine.
A question naturally arises: would it be possible to convert any Markdown file, no matter how sadistic, to PDF? . suppose As a practical matter, I'm sure such things exist already. I have no interest in being in the Markdown-conversion business. As a theoretical matter: the problem I foresee is that Markdown —like the HTML that it lightly disguises —can have formatting entities that are nested indefinitely deep. The idea of making a layout engine that can handle that just becomes the equivalent of reinventing a web-browser engine.
Curious characters can do this:
Back to the demo. Curious characters can do this:
@terminal{
> doc
@ -223,24 +253,36 @@ Curious characters can do this:
This is part of the @tech{Q-expression} that the source file produces. This Q-expression is passed to Quadwriter for layout and rendering.
If you know about the duality of @tech{X-expressions} and XML, you might wonder if we could create an equivalent text-based markup language for Q-expressions —let's call it QML. Sure, why not:
This is the Q-expression that the source file produces. This Q-expression is passed to Quad for layout and rendering.
There's nothing interesting about QML —it's just a way of cosmetically encoding an S-expression as a string. We could also convert our Q-expression to, say, JSON. Thus, having noted that QML can exist, and @racket[quadwriter] could support this input, let's move on.
@subsection{Hard rock: Quadwriter + markup}
@subsection{Hard rock: Quadwriter & markup}
Suppose Markdown is just not your thing. You prefer to enter your markup the old-fashioned way —by hand. I hear you. So let's switch to the @racket[quadwriter/markup] dialect. First we try our simple test:
@ -275,7 +317,7 @@ And they love to code
}|
]
The special @litchar{◊} character is called a @defterm{lozenge}. It introduces markup tags. @link["https://docs.racket-lang.org/pollen/pollen-command-syntax.html#%28part._the-lozenge%29"]{Instructions for typing it}, but for now it suffices to copy & paste, or use the @onscreen{Insert Command Char} button in the DrRacket toolbar.
The special @litchar{◊} character is called a @deftech{lozenge}. It introduces markup tags. @link["https://docs.racket-lang.org/pollen/pollen-command-syntax.html#%28part._the-lozenge%29"]{Instructions for typing it}, but for now it suffices to copy & paste, or use the @onscreen{Insert Command Char} button in the DrRacket toolbar.
Under the hood, the @racket[quadwriter/markdown] dialect is converting the Markdown surface notation into markup tags that look like this. So the @racket[quadwriter/markup] dialect just lets us start with those tags.
@ -287,164 +329,185 @@ Curious characters can prove that this is so by again typing at the REPL:
This Q-expression is exactly the same as the one that resulted with the @racket[quadwriter/markdown] source file.
@racket[quadwriter/markdown] showed high-level notation (= a generous way of describing Markdown) that generated a Q-expression. Then @racket[quadwriter/markup] showed a mid-level notation that generated another (identical) Q-expression.
@racket[quadwriter/markdown] showed a high-level notation that generated a Q-expression. Then @racket[quadwriter/markup] showed a mid-level notation that generated another (identical) Q-expression.
If we wish, we can also skip the notational foofaraw and just write Q-expressions directly in our source file. We do this with the basic @racket[quadwriter] language.
If we wish, we can also skip the notational layers and just write Q-expressions directly in our source file. We do this with the basic @racket[quadwriter] language. As usual, first comes our simple test:
Recall our very first example:
@fileblock["test.rkt"
@codeblock|{
#lang quadwriter
#lang quadwriter/markup
Brennan and Dale like fancy sauce.
}|
]
The result is the same as before.
@;{
This is the compiled Quadwriter markup, showing what is sent to the typesetting engine.
@code{#lang quadwriter} is a Racket-implemented DSL (= domain-specific language). It's not a language in the sense of Turing-complete. Rather, a @code{#lang quadwriter} ``program'' resembles text annotated with high-level layout-description commands (not unlike XML/HTML). @code{#lang quadwriter} programs can be written directly, or generated as the output of other programs.
Each @"@"-expression in @code{#lang quadwriter} is interpreted as a @italic{quad} (roughly a box; more precisely a contiguous formatting region). A quad has the following syntax:
In the REPL, the @racket[doc] was this Q-expression:
The @code{(list 'attr-name attr-val ...)} is an interleaved list of symbols and values, as you might provide to @racket[hash]. The attribute list is mandatory. If a quad has no attributes of its own, this can be signaled with either @racket[empty] or @racket[#f]:
@codeblock|{
@quad-name[empty]{text-or-more-quads ...}
@quad-name[#f]{text-or-more-quads ...}
}|
If you thought this resembled an @link["http://docs.racket-lang.org/pollen/second-tutorial.html#%28part._.X-expressions%29"]{X-expression}, you wouldn't be wrong. Like X-expressions, quads are recursively composable. Also like X-expressions, the attributes in a quad apply to all the text or quads within, unless superseded by another attribute declaration deeper down.
Let's see how this works. The simplest kind of quad is a @code{block}. If we wrap our text in a @code{block} without attributes, what happens to the PDF?
@codeblock|{
#lang quadwriter
@block[#f]{Brennan and Dale like fancy sauce.}
}|
@repl-output{
'(q ((line-height "17")) (q ((break "paragraph"))) "Brennan and Dale like fancy sauce." (q ((break "paragraph"))))
}
Right —nothing. A block without attributes just evaporates. Move the boundaries of the block:
Let's copy this Q-expression and use it as our new source code. This time, however, we'll switch to plain @code{#lang quadwriter} (instead of the @racket[markup] or @racket[markdown] dialects):
"Brennan and Dale like fancy sauce." (q ((break "paragraph"))))
}|
]
Still the same. Let's add some bold formatting with the @code{weight} attribute:
This produces the same one-line PDF as before.
@codeblock|{
#lang quadwriter
@block['(weight bold)]{Brennan and Dale} like fancy sauce.
}|
Likewise, we can pick up the @racket[doc] from our more complex example:
What an accomplishment. To show you that attributes are additive, we'll put a quad inside our quad:
@codeblock|{
#lang quadwriter
@block['(weight bold)]{Brennan and @block['(color "red")]{Dale}} like fancy sauce.
}|
You're getting the idea. In terms of type styling, here are the attributes and values that Quad understands:
@code{'weight} = @code{'normal} (default) or @code{'bold}
@(linebreak)@code{'style} = @code{'normal} (default) or @code{'italic}
@(linebreak)@code{'font} = family name as string
@(linebreak)@code{'size} = point size as floating-point number
@(linebreak)@code{leading} = baseline-to-baseline measure in points
@(linebreak)@code{'color} = color string from @racket[color-database<%>]
Feel free to impose these on your demo program.
#lang quadwriter/markdown
# Did you know?
Though we're using @"@"-expressions, a @code{#lang quadwriter} source file doesn't imply any formatting characteristics as it would in Scribble or Pollen. For instance, see what happens if you add two line breaks and some more text:
__Brennan__ and **Dale** like:
@codeblock|{
#lang quadwriter
@block['(size 16)]{Brennan and @block['(color "red")]{Dale}} like fancy sauce.
* *Fancy* sauce
* _Chicken_ fingers
Derek does not.
```
And they love to code
```
}|
The text ``Derek does not'' appears flush against the first sentence. In Scribble those linebreaks would suggest a paragraph break. In HTML they would suggest a word space. In @code{#lang quadwriter} they suggest neither. Why not? Because @code{#lang quadwriter} is strictly a language for describing explicit typesetting. Newlines have no meaning.
OK, so how do we create a paragraph? Quad supports a special set of quads called @italic{breaks} that move the typesetting position. For instance, @code{@"@"(block-break)} will act like a carriage return, moving the next typeset item below the previous item and all the way to the left edge of the column:
And again, use this Q-expression as the source for a new @racket[quadwriter] program:
@fileblock["test.rkt"
@codeblock|{
#lang quadwriter
@block['(size 16)]{Brennan and @block['(color "red")]{Dale}} like fancy sauce.
Now ``Derek does not'' appears on its own line. What about things like paragraph spacing and first-line indents? Again, because @code{#lang quadwriter} is about explicit typesetting, all these things need to be inserted explicitly in the code. For instance, to make an indent, we add a @code{box} with a @code{'width} attribute:
Which yields the same PDF result. (If you've spent any time with ’90s HTML markup, the above probably looks familiar.)
@codeblock|{
#lang quadwriter
@block['(size 16)]{Brennan and @block['(color "red")]{Dale}} like fancy sauce.
@(block-break)
@box['(width 15)]
Derek does not.
}|
@subsection{Q-expression PS}
Quad also handles @code|{@(line-break)}|, @code|{@(column-break)}|, and @code|{@(page-break)}|. Try the last one:
In @code{#lang quadwriter}, we enter our Q-expression in the usual Racket list notation. What if we wanted to use the text-based notation of @racket[quadwriter/markup]? Sure —we can convert our Q-expression to that notation:
@fileblock["test.rkt"
@codeblock|{
#lang quadwriter
@block['(size 16)]{Brennan and @block['(color "red")]{Dale}} like fancy sauce.
#:font-size "10" #:bg "aliceblue"]{And they love to
code}}
◊q[#:break "paragraph"]}
}|
]
Next, let's look at Quad's linebreaking mechanism. For the next sample, please paste in a large block of plain text between the curly braces so you'll get some linewrapping:
Don't panic —you'll never have to actually do this in @racket[quadwriter]. It's just to show that it's possible.
@codeblock|{
#lang quadwriter
@block[#f]{A text that goes on for a while ...}
}|
@subsection{Quadwriter: the upshot}
You will see a block of justified text. The demo is running at maximum quality, so two other things will also be true.
What you should infer from all this is that in the usual Racket tradition, @racket[quadwriter] and its dialects are just compiling a document from a higher-level representation to a lower-level representation.
First, the lines are broken using the Knuth-Plass algorithm developed for TeX. This is a very nice algorithm that looks at all possible ways of breaking the lines in the paragraph and picks the one that leaves the smallest total gap at the right edge. (Quad will also hyphenate if necessary, but only if all the unhyphenated possibilities are bad.)
If you're a writer, you might prefer to use the high-level representation (like Markdown) so that your experience is optimized for ease of use.
Second, notice that punctuation hangs a little outside the text block. This is an optical adjustment that makes for neater-looking blocks. Whether you literally care about this kind of optical adjustment is not the point. The point is that the Quad typesetting engine @italic{permits} it. And that is really what we are going for here: a hackable typesetting engine. When you have fine control over all the page elements, then other things become possible (for instance, mathematical-equation typesetting, which is quite a bit more involved than just hanging punctuation off the edges).
If you're a developer, you might prefer to use the lower-level representation for precision. For instance, a @racketmodname[pollen] author who wanted to generate a PDF could design tag functions that emit Q-expressions, and then pass the result to Quadwriter for conversion to PDF.
Justification is the default setting for the demo. To override this setting, use these attributes:
Or, you can aim somewhere in between. Like everything else in Racket, you can design functions & macros to emit the pieces of a Q-expression using whatever interface you prefer.
@code{'x-align} = @code{'justify} (default), @code{'left}, @code{'center}, or @code{'right}
@code{'x-align-last-line} = @code{'justify} (default), @code{'left}, @code{'center}, or @code{'right}
@subsection{I don't like Quadwriter}
It's a demo! Don't panic! @racket[quadwriter] itself is just meant to show how one can build an interface to @racket[quad], which if we're being honest, is basically just a home for all the generic geometric routines and technical fiddly bits (e.g., font parsing and PDF generation) without any true typographic smarts. That's what @racket[quadwriter] adds.
@codeblock|{
#lang quadwriter
@block['(x-align center x-align-last-line center)]{A text that goes on for a while ...}
}|
In the sample above, though you see formatting tags like @racket[background-color] and @racket[font-family], those are defined by @racket[quadwriter], not @racket[quad]. So if you don't like them —no problem! You can still drop down one more layer and program your own interface to @racket[quad].
Then you can combine blocks with different styles:
That said, I imagine that most users & developers are looking for PDF generation along the lines of ``don't make me think too hard.'' So I can foresee that @racket[quadwriter] (or a better replacement) will be the preferred interface.
@codeblock|{
#lang quadwriter
@block['(x-align center x-align-last-line center size 24)]{Very important headline}
@(block-break)
@block['(style italic)]{A subhead that maybe lasts a few lines}
@(block-break)
@box['(width 10)]@block[#f]{A text that goes on for a while ...}
}|
Why? Decades of experience with HTML and its relations have acclimated us to the model of marking up a text with certain codes that denote layout, and then passing the markup to a program for output. So I think the idea of a Q-expression, and some vocabulary of markup tags will probably end up being the most natural and useful interface.
In sum, you can build up complex typesetting with a relatively small vocabulary of typesetting commands.
@margin-note{Historians of desktop word processors may remember that WordPerfect has a @onscreen{Reveal codes} feature which lets you drop into the markup that represents the formatting displayed in the GUI.}
You are welcome to shovel large quantities of plain text into your @code{#lang quadwriter} window to see it broken into lines and paginated.
}
@section{Why is it called Quad?}
In letterpress printing, a @italic{quad} was a piece of metal used as spacing material within a line.
@section{Why is it called Quad?}
In letterpress printing, a @italic{quad} was a piece of metal used as spacing material within a line.
@italic{``A way of doing something original is by trying something
so painstaking that nobody else has ever bothered with it.'' —Brian Eno}
