@italic{Parts of this section were adapted from Matthew Flatt and Eli Barzilay's excellent documentation for Racket's text-processing language, called Scribble◊.}
@section{The golden rule}
@section{The golden rule}
@ -54,7 +52,7 @@ A text-mode command has the three possible parts after the @litchar["◊"]:
@itemlist[
@itemlist[
@item{The @italic{command name} appears immediately after the @litchar["◊"]. Typically it's a short word.}
@item{The @italic{command name} appears immediately after the @litchar["◊"]. Typically it's a short word.}
@item{The @italic{Racket arguments} appear between square brackets. Pollen is partly an interface to the Racket programming language. These arguments have to be entered using Racket conventions —e.g., a @tt{string of text} needs to be put in quotes as a @code{"string of text"}. If you like programming, you'll end up using these frequently. If you don't, you won't.}
@item{The @italic{Racket arguments} appear between square brackets. Pollen is partly an interface to the Racket programming language. These arguments are entered using Racket conventions —e.g., a @tt{string of text} needs to be put in quotes as a @code{"string of text"}. If you like programming, you'll end up using these frequently. If you don't, you won't.}
@item{The @italic{text argument} appears between braces (aka curly brackets). You can put any ordinary text here. Unlike with the Racket arguments, you don't put quotes around the text.}
@item{The @italic{text argument} appears between braces (aka curly brackets). You can put any ordinary text here. Unlike with the Racket arguments, you don't put quotes around the text.}
]
]
@ -422,690 +420,214 @@ The value of edge is ◊|edge| pixels}
limited to identifiers; it can be @italic{any} Racket expression.
@subsection{The Racket arguments}
@scribble-examples|==={
The middle part of a text-mode Pollen command contains the @italic{Racket arguments} @litchar{[}between square brackets.@litchar{]} Most often, you'll see these used to pass extra information to commands that operate on text.
@(lambda (x) x){blah}
@`(unquote foo){blah}
}===|
In addition, the command can be omitted altogether, which will omit it
For instance, tag functions. Recall from before that any not-yet-defined command name in Pollen is treated as a tag function:
from the translation, resulting in an S-expression that usually
contains, say, just strings:
@scribble-examples|==={
@codeblock|{
@{foo bar
#lang pollen
baz}
◊title{The Beginning of the End}
@'{foo bar
}|
baz}
}===|
If the command part begins with a @litchar{;} (with no newline between
@racketoutput{@literal{'(title "The Beginning of the End")}}
the @litchar["@"] and the @litchar{;}), then the construct is a
comment. There are two comment forms, one for arbitrary-text and
possibly nested comments, and another one for line comments:
@racketblock[
But what if you wanted to add attributes to this tag, so that it comes out like this?
@racketoutput{@literal{'(title ((class "red")(id "first")) "The Beginning of the End")}}
]
In the first form, the commented body must still parse correctly; see
You can do it with Racket arguments.
the description of the body syntax below. In the second form, all
text from the @litchar["@;"] to the end of the line @italic{and} all
following spaces (or tabs) are part of the comment (similar to
@litchar{%} comments in TeX).
@scribble-examples|==={
Here's the hard way. You can type out your list of attributes in Racket format and drop them into the brackets as a single argument:
@foo{bar @; comment
baz@;
blah}
}===|
Tip: if you use an editor in some Scheme mode without support for
@codeblock|{
@tech{◊-forms}, balanced comments can be confusing, since the open brace
#lang pollen
looks commented out, and the closing one isn't. In such cases it is
◊title['((class "red")(id "first"))]{The Beginning of the End}
useful to ``comment'' out the closing brace too:
}|
@racketoutput{@literal{'(title ((class "red") (id "first")) "The Beginning of the End")}}
But that's a lot of parentheses to think about. So here's the easy way. Anytime you use a tag function, there's a shortcut for inserting attributes. You can enter them as a series of @racket[symbol] / @racket[string] pairs between the Racket-argument brackets. The only caveat is that the symbols have to begin with a quote mark @litchar{'} and end with a colon @litchar{:}. So taken together, they look like this:
@codeblock|{
#lang pollen
◊title['class: "red" 'id: "first"]{The Beginning of the End}
}|
@verbatim[#:indent 2]|==={
@racketoutput{@literal{'(title ((class "red") (id "first")) "The Beginning of the End")}}
@;{
...
;}
}===|
so the editor does not treat the file as having unbalanced
Racket arguments can be any valid Racket expressions. For instance, this will also work:
parentheses.
If only the @nonterm{cmd} part of an @tech{◊-form} is specified, then the
@codeblock|{
result is the command part only, without an extra set of parenthesis.
#lang pollen
This makes it suitable for Racket escapes in body texts. (More on this
◊title['class: (format "~a" (* 6 7)) 'id: "first"]{The Beginning of the End}
below, in the description of the body part.)
}|
@scribble-examples|==={
@racketoutput{@literal{'(title ((class "42") (id "first")) "The Beginning of the End")}}
@foo{x @y z}
@foo{x @(* y 2) z}
@{@foo bar}
}===|
Finally, note that there are currently no special rules for using
Since Pollen commands are really just Racket arguments underneath, you can use those too. Here, we'll define a variable called @tt{name} and use it in the Racket arguments of @tt{title}:
@litchar["@"] in the command itself, which can lead to things like:
@scribble-examples|==={
@codeblock|{
@@foo{bar}{baz}
#lang pollen
}===|
◊(define name "Brennan")
◊title['class: "red" 'id: ◊name]{The Beginning of the End}
}|
@racketoutput{@literal{'(title ((class "read") (id "Brennan")) "The Beginning of the End")}}
You can also use this area for @italic{keyword arguments}. Keyword arguments can be used to provide options for a particular Pollen command, to avoid redundancy. Suppose that instead of using the @tt{h1 ... h6} tags, you want to consolidate them into one command called @tt{heading} and select the level separately. You can do this with a keyword, in this case @racket[#:level], which is passed as a Racket argument:
The example shows how an input syntax is read as Racket syntax, not
The third part of a text-mode Pollen command is the text argument. The text argument @litchar{@"{"}appears between curly braces@litchar{@"}"}. It can contain any text you want. The text argument can also contain other Pollen commands with their own text arguments. And they can contain other Pollen commands ... and so on, all the way down.
what it evaluates to. If you want to see the translation of an example
into S-expression form, add a quote in front of it in a
@at-exp-racket[] module. For example, running
@verbatim[#:indent 2]|{
@codeblock|{
#lang at-exp racket
#lang pollen
'@foo{blah blah blah}
◊div{Do it again. ◊div{And again. ◊div{And yet again.}}}
Three small details to know about the text argument.
while omitting the quote
First, the only character that needs special handling in a text argument is the lozenge @litchar{◊}. A lozenge ordinarily marks a new command. So if you want an actual lozenge to appear in the text, you have to escape it by typing @litchar{◊"◊"}.
@verbatim[#:indent 2]|{
@codeblock|{
#lang at-exp racket
#lang pollen
@foo{blah blah blah}
◊definition{This is the lozenge: ◊"◊"}
}|
}|
triggers a syntax error because @racket[foo] is not bound, and
@racketoutput{@literal{'(definition "This is the lozenge: ◊")}}
Second, the whitespace-trimming policy. Here's the short version: if there's a carriage return at either end of the text argument, it is trimmed, and whitespace at the end of each line is selectively trimmed in an intelligent way. So this text argument, with carriage returns on the ends:
@verbatim[#:indent 2]|{
@codeblock|{
#lang at-exp racket
#lang pollen
(define (foo str) (printf "He wrote ~s.\n" str))
◊div{
@foo{blah blah blah}
Roomy!
I agree.
}
}|
}|
prints the output
@racketoutput{@literal{'(div "Roomy!" "\n" "\n" "I agree.")}}
@nested[#:style 'inset]{@racketoutput{He wrote "blah blah blah".}}
Here are more examples of @tech{◊-forms}:
@scribble-examples|==={
@foo{blah "blah" (`blah'?)}
@foo[1 2]{3 4}
@foo[1 2 3 4]
@foo[#:width 2]{blah blah}
@foo{blah blah
yada yada}
@foo{
blah blah
yada yada
}
}===|
As seen in the last example, multiple lines and the newlines that
separate them are parsed to multiple Racket strings. More generally,
a @nonterm{text-body} is made of text, newlines, and nested
@tech{◊-forms}, where the syntax for @tech{◊-forms} is the same whether it's
in a @nonterm{text-body} context as in a Racket context. A
@nonterm{text-body} that isn't an @tech{◊-form} is converted to a string
expression for its @nonterm{parsed-body}; newlines and following
indentations are converted to @racket["\n"] and all-space string
expressions.
@scribble-examples|==={
@foo{bar @baz{3}
blah}
@foo{@b{@u[3] @u{4}}
blah}
@C{while (*(p++))
*p = '\n';}
}===|
The command part of an @tech{◊-form} is optional as well. In that case,
the @tech{◊-form} is read as a list, which usually counts as a function
application, but it also useful when quoted with the usual Racket
@racket[quote]:
@scribble-examples|==={
@{blah blah}
@{blah @[3]}
'@{foo
bar
baz}
}===|
Finally, we can also drop the datum and text parts, which leaves us with
only the command---which is read as is, not within a parenthesized
form. This is not useful when reading Racket code, but it can be used
inside a text block to escape a Racket identifier. A vertical bar
(@litchar{|}) can be used to delimit the escaped identifier when
needed.
@scribble-examples|==={
@foo
@{blah @foo blah}
@{blah @foo: blah}
@{blah @|foo|: blah}
}===|
Actually, the command part can be any Racket expression (that does not
start with @litchar["["], @litchar["{"], or @litchar["|"]), which is
particularly useful with such escapes since they can be used with any
expression.
@scribble-examples|==={
@foo{(+ 1 2) -> @(+ 1 2)!}
@foo{A @"string" escape}
}===|
Note that an escaped Racket string is merged with the surrounding text
as a special case. This is useful if you want to use the special
characters in your string, but escaping braces are not necessary if
they are balanced.
@scribble-examples|==={
@foo{eli@"@"barzilay.org}
@foo{A @"{" begins a block}
@C{while (*(p++)) {
*p = '\n';
}}
}===|
In some cases, a text contains many literal ◊s, which can be
cumbersome to quote individually. For such case, braces have an
alternative syntax: A block of text can begin with a
``@litchar["|{"]'' and terminated accordingly with a
``@litchar["}|"]''. Furthermore, any nested @tech{◊-forms} must begin
with a ``@litchar["|@"]''.
@scribble-examples|==={
@foo|{bar}@{baz}|
@foo|{bar |@x{X} baz}|
@foo|{bar |@x|{@}| baz}|
}===|
In cases when even this is not convenient enough, punctuation
characters can be added between the @litchar{|} and the braces and the
◊ in nested forms. (The punctuation is mirrored for parentheses
and @litchar{<>}s.) With this extension, Pollen syntax can be used as a
``here string'' replacement.
@scribble-examples|==={
@foo|--{bar}@|{baz}--|
@foo|<<{bar}@|{baz}>>|
}===|
On the flip side of this is, how can an ◊ sign be used in Racket
code? This is almost never an issue, because Racket strings and
characters are still read the same, and @litchar["@"] is set as a
non-terminating reader macro so it can be used in Racket identifiers
anywhere except in the first character of an identifier. When
@litchar["@"] must appear as the first character of an identifier, you
must quote the identifier just like other non-standard characters in
normal S-expression syntax: with a backslash or with vertical bars.
@scribble-examples|==={
(define \@email "foo@bar.com")
(define |@atchar| #\@)
}===|
Note that spaces are not allowed before a @litchar{[} or a
@litchar["{"], or they will be part of the following text (or Racket
code). (More on using braces in body texts below.)
@scribble-examples|==={
@foo{bar @baz[2 3] {4 5}}
}===|
Finally, remember that the Pollen is just an alternate for
S-expressions. Identifiers still get their meaning, as in any
Racket code, through the lexical context in which they appear.
Specifically, when the above @tech{◊-form} appears in a Racket expression
context, the lexical environment must provide bindings for
@racket[foo] as a procedure or a macro; it can be defined, required,
or bound locally (with @racket[let], for example).
The datum part can contains arbitrary Racket expressions, which
I agree.}
are simply stacked before the body text arguments:
}|
@scribble-examples|==={
@racketoutput{@literal{'(div "Roomy!" "\n" "\n" "I agree.")}}
@foo[1 (* 2 3)]{bar}
@foo[@bar{...}]{blah}
}===|
The body part can still be omitted, which is essentially an
For the long version, please see @secref{Spaces, Newlines, and Indentation}.
alternative syntax for plain (non-textual) S-expressions:
@scribble-examples|==={
@foo[bar]
@foo{bar @f[x] baz}
}===|
The datum part can be empty, which makes no difference, except when
Third, within a multiline text argument, newline characters become individual strings that are not merged with adjacent text. So what you end up with is a list of strings, not a single string. That's why in the last example, we got this:
the body is omitted. It is more common, however, to use an empty body
for the same purpose.
@scribble-examples|==={
@racketoutput{@literal{'(div "Roomy!" "\n" "\n" "I agree.")}}
@foo[]{bar}
@foo[]
@foo
@foo{}
}===|
The most common use of the datum part is for Racket forms that expect
Instead of this:
keyword-value arguments that precede the body of text arguments.
Under most circumstances, these two tagged X-expressions will behave the same way. The biggest exception is with functions. A function that operates on multiline text arguments needs to be able to handle an indefinite number of strings. For instance, this @tt{jejune} function only accepts a single argument. It will work with a single-line text argument, because that produces a single string:
@section{The Body Part}
The syntax of the body part is intended to be as convenient as
possible for free text. It can contain almost any text---the only
characters with special meaning is @litchar["@"] for sub-@tech{◊-forms},
and @litchar["}"] for the end of the text. In addition, a
@litchar["{"] is allowed as part of the text, and it makes the
matching @litchar["}"] be part of the text too---so balanced braces
are valid text.
@scribble-examples|==={
@foo{f{o}o}
@foo{{{}}{}}
}===|
As described above, the text turns to a sequence of string arguments
for the resulting form. Spaces at the beginning and end of lines are
discarded, and newlines turn to individual @racket["\n"] strings
(i.e., they are not merged with other body parts); see also the
information about newlines and indentation below. Spaces are
@italic{not} discarded if they appear after the open @litchar["{"]
(before the closing @litchar["}"]) when there is also text that
follows (precedes) it; specifically, they are preserved in a
single-line body.
@scribble-examples|==={
@foo{bar}
@foo{ bar }
@foo[1]{ bar }
}===|
If @litchar["@"] appears in a body, then it is interpreted as Racket
code, which means that the ◊-reader is applied recursively, and the
resulting syntax appears as part of the S-expression, among other
string contents.
@scribble-examples|==={
@foo{a @bar{b} c}
}===|
If the nested ◊ construct has only a command---no body or datum
parts---it will not appear in a subform. Given that the command part
can be any Racket expression, this makes ◊ a general escape to
arbitrary Racket code.
@scribble-examples|==={
@foo{a @bar c}
@foo{a @(bar 2) c}
}===|
This is particularly useful with strings, which can be used to include
arbitrary text.
@scribble-examples|==={
@foo{A @"}" marks the end}
}===|
Note that the escaped string is (intentionally) merged with the rest
But watch what happens with a multiline text argument:
@subsection{Comments}
@codeblock|{
#lang pollen
◊(define (jejune text)
`(jejune ,text))
◊jejune{Deeply
chastened}
}|
@racketerror{jejune: arity mismatch;@(linebreak)
the expected number of arguments does not match the given number@(linebreak)
expected: 1@(linebreak)
given: 3@(linebreak)
arguments...:@(linebreak)
"Deeply"@(linebreak)
"\n"@(linebreak)
"chastened"}
As noted above, there are two kinds of Pollen comments: @litchar|{@;{...}}| is
The answer is to use a @italic{rest argument} in the function, which takes the ``rest'' of the arguments —however many there may be —and combines them into a single @racket[list]. If we rewrite @tt{jejune} with a rest argument, we can fix the problem:
a (nestable) comment for a whole body of text (following the same
rules for @tech{◊-forms}), and @litchar|{@;...}| is a line-comment.
@scribble-examples|==={
@codeblock|{
@foo{First line@;{there is still a
#lang pollen
newline here;}
◊(define (jejune . texts)
Second line}
`(jejune ,@texts))
}===|
◊jejune{Deeply
chastened}
}|
One useful property of line-comments is that they continue to the end
The Pollen language is a variant of Racket's own text-processing language, called Scribble. So many things that are true about Scribble are also true about Pollen. For the sake of clarity & brevity, I've omitted them from this summary. But if you want the full story:
@subsection{Spaces, Newlines, and Indentation}
The Pollen syntax treats spaces and newlines in a special way is
meant to be sensible for dealing with text. As mentioned above,
spaces at the beginning and end of body lines are discarded, except
for spaces between a @litchar["{"] and text, or between text and a
@litchar["}"].
@scribble-examples|==={
@foo{bar}
@foo{ bar }
@foo{ bar
baz }
}===|
A single newline that follows an open brace or precedes a closing
brace is discarded, unless there are only newlines in the body; other
newlines are read as a @racket["\n"] string
@scribble-examples|==={
@foo{bar
}
@foo{
bar
}
@foo{
bar
}
@foo{
bar
baz
}
@foo{
}
@foo{
}
@foo{ bar
baz }
}===|
Spaces at the beginning of body lines do not appear in the resulting
S-expressions, but the column of each line is noticed, and all-space
indentation strings are added so the result has the same indentation.
A indentation string is added to each line according to its distance
from the leftmost syntax object (except for empty lines). (Note: if
you try these examples on a Racket REPL, you should be aware that
the reader does not know about the ``@litchar{> }'' prompt.)
@scribble-examples|==={
@foo{
bar
baz
blah
}
@foo{
begin
x++;
end}
@foo{
a
b
c}
}===|
If the first string came from the opening @litchar["{"] line, it is
not prepended with an indentation (but it can affect the leftmost
syntax object used for indentation). This makes sense when formatting
structured code as well as text (see the last example in the following
block).
@scribble-examples|==={
@foo{bar
baz
bbb}
@foo{ bar
baz
bbb}
@foo{bar
baz
bbb}
@foo{ bar
baz
bbb}
@foo{ bar
baz
bbb}
@text{Some @b{bold
text}, and
more text.}
}===|
Note that each ◊-form is parsed to an S-expression that has its own
indentation. This means that Pollen source can be indented like
code, but if indentation matters then you may need to apply
indentation of the outer item to all lines of the inner one. For
example, in
@litchar/lines|==={
@code{
begin
i = 1, r = 1
@bold{while i < n do
r *= i++
done}
end
}
}===|
a formatter will need to apply the 2-space indentation to the
rendering of the @racket[bold] body.
Note that to get a first-line text to be counted as a leftmost line,
line and column accounting should be on for the input port
(@racket[use-at-readtable] turns them on for the current input port).
Without this,
@litchar/lines|==={
@foo{x1
x2
x3}
}===|
will not have 2-space indentations in the parsed S-expression if
source accounting is not on, but
@litchar/lines|==={
@foo{x1
x2
x3}
}===|
will (due to the last line). Pay attention to this, as it can be a
problem with Racket code, for example:
@litchar/lines|==={
@code{(define (foo x)
(+ x 1))}
}===|
For rare situations where spaces at the beginning (or end) of lines
matter, you can begin (or end) a line with a @litchar["@||"].
