One curious aspect of free software is that you can appropriate the benefits of other people’s work while making it look like your own. No such legerdemain here. Whatever effort I’ve put into Pollen is dwarfed by the epic accomplishments of the Racket development team. I thank all of them — especially Matthew Flatt, Jay McCarthy, and Matthias Felleisen — for making this tremendous tool available, for adding several features I suggested, and for patiently answering my dumb questions over the months.
Thank you to Greg Hendershott for his Markdown parser.
The best software tools do more than get the job done. They create an incentive to undertake jobs you wouldn’t have attempted before. Racket encouraged me to become a better programmer so I could create Pollen. Likewise, I hope that Pollen encourages you to make things you couldn’t before.
One curious aspect of free software is that you can appropriate the benefits of other people’s work while making it look like your own. No such legerdemain here. Whatever effort I’ve put into Pollen is dwarfed by the epic accomplishments of the Racket development team. I thank all of them — especially Matthew Flatt, Jay McCarthy, and Matthias Felleisen — for making this tremendous tool available, for adding several features I suggested, and for patiently answering my dumb questions over the months.
Thank you to Greg Hendershott for his Markdown parser.
The best software tools do more than get the job done. They create an incentive to undertake jobs you wouldn’t have attempted before. Racket encouraged me to become a better programmer so I could create Pollen. Likewise, I hope that Pollen encourages you to make things you couldn’t before.
\ No newline at end of file
diff --git a/doc/Backstory.html b/doc/Backstory.html
index 49e1f17..e43bed1 100644
--- a/doc/Backstory.html
+++ b/doc/Backstory.html
@@ -1,2 +1,2 @@
-3 Backstory
I created Pollen to overcome limitations & frustrations I repeatedly encountered with existing web-publishing tools.
If you agree with my characterization of those problems, then you’ll probably like the solution that Pollen offers. If not, you probably won’t.
3.1Web development and its discontents
I made my first web page in 1994, shortly after the web was invented. I opened my text editor (at the time, BBEdit), pecked out <html><body>Helloworld</body></html>, then loaded it in Mosaic. So did a million other nerds.
If you weren’t around then, you didn’t miss much. Everything about the web was horrible: the web browsers, the computers running the browsers, the dial-up connections feeding the browsers, and of course HTML itself. At that point, the desktop-software experience was already slick and refined. By comparison, using the web felt like banging rocks together.
That’s no longer true. The web is now 20 years old. During that time, most parts of the web have improved dramatically — for instance, the connections are faster, the browsers are more sophisticated, the screens have more pixels.
But one part has not improved: the way we make web pages. Over the years, tools promising to simplify HTML development have come and mostly gone — from PageMill to Dreamweaver to WordPress to Jekyll. Meanwhile, true web jocks have remained loyal to the original HTML power tool: the humble text editor.
In one way, this makes sense. Web pages are mostly made of text-based data — HTML, CSS, JavaScript, and so on — and the simplest way to mainpulate this data is with a text editor. While HTML and CSS are not programming languages, they lend themselves to semantic and logical structure that’s most easily expressed by editing them as text. Furthermore, text-based editing makes debugging and performance improvements easier.
But text-based editing is also limited. Though the underlying description of a web page is notionally human-readable, it’s largely optimized to be readable by other software — namely, web browsers. HTML markup in particular is verbose and easily mistyped. And isn’t it fatally dull to manage all the boilerplate, like surrounding every paragraph with <p>...</p>? Yes, it is.
For these reasons, much of web development should lend itself to abstraction & automation. Abstraction means consolidating repetitve, complex patterns into simpler, parameterized forms. Automation means avoiding the manual drudgery of generating the output files. But in practice, tools that enable this abstraction & automation have been slow to arrive, and most have come hobbled with unacceptable deficiencies.
3.2The better idea: a programming model
Parallel with my HTML education, I also goofed around with various programming languages — C, C++, Perl, Java, PHP, JavaScript, Python. Unlike HTML, programming languages excel at abstraction and automation. This seemed like the obvious direction for web development to go.
What distinguishes the text-editing model from the programming model? It’s a matter of direct vs. indirect manipulation of output. The text-editing model treats HTML as something to be written directly with a text editor. Whereas the programming model treats HTML — or whatever the output is — as the result of compiling a set of source files, which are written in a programming language. The costs of working indirectly via the programming language are offset by the benefits of abstraction & automation.
On the early web, the text-editing model was appealingly precise and quick. On small projects, it worked well enough. But as projects grew, the text-editing model was going to lose steam. I wasn’t the only one to notice. Shortly after those million nerds made their first web page by hand, many of them set about devising ways to apply a programming model to web development.
3.3“Now you have two problems”
What followed was a steady stream of products, frameworks, tools, and content management systems that claimed to bring a programming model to web development. Some were better than others. But none of them displaced the text editor as the preferred tool of web developers.
Why not? All these tools promised a great leap forward in solving the web-development problem. In practice, they simply redistributed the pain. I needn’t bore you with enumerating the deficiencies of specific tools, because they’ve tended to fail in the same thematic ways:
No native data structure for HTML. Core to any programming model is data structures. Good data structures make processing easy; bad ones make it hard. Even though HTML has a well documented format, rarely has it been handled within a programming system with a native, intuitive data structure. Instead, it’s either been treated as a string (wrong), a tree (also wrong), or some magical parsed object. This has made working with HTML in programming environments needlessly difficult.
Mandatory separation of code, presentation, and content. This principle has often been held out as an ideal in web development. But it’s also counterintuitive, because an HTML page naturally contains all three. If you want to separate them, your tools should let you. But if you don’t, your tools shouldn’t make you.
Compromised template languages. Seems like every programming language has at least 10 templating systems for HTML, all of which require you to learn a new “template language” that offers the worst of both worlds: fewer features and different syntax than the underlying language.
Steep learning curves. Web programmers have often chided designers for not knowing how to code. But programming-based web-development tools have often had a high initial learning curve that requires you to throw out your existing workflow. Programmers built these tools — no surprise that programmers have been more comfortable with them.
I’ve tried a lot of these tools over the years. Some I liked. Some I didn’t. Invariably, however, whenever I could still make do with hand-editing an HTML project, I would. After trying to cajole the web framework du jour into doing my bidding, it was relaxing to trade off some efficiency for control.
3.4Rethinking the solution for digital books
In 2008, I launched a website called Typography for Lawyers. Initially, I’d conceived of it as a book. Then I thought “no one’s going to publish that.” So it became a website, that I aimed to make as book-like as possible. But hand-editing wasn’t going to be enough.
So I used WordPress. The major chore became scraping out all the crap that typically lives in blog templates. Largely because of this, people liked the site, because it was simpler & cleaner than the usual WordPress website.
Eventually, a publisher offered to release it as a paperback. Later came the inevitable request to make it into a Kindle book. As a fan of typography, I hate the Kindle. The layout controls are coarse, and so is the reading experience. But I didn’t run and hide. Basically a Kindle book is a little website made with 1995-era HTML. So I coded up some tools in Perl to convert my book to Kindle format while preserving the formatting and images as well as possible.
At that point, I noticed I had converted Typography for Lawyers into web format twice, using two different sets of tools. Before someone asked me to do it a third time, I started thinking about how I might create source code for the book that allowed me to render it into different formats.
This was the beginning of the Pollen project.
I wrote the initial version of Pollen in Python. I devised a simplified markup-notation language for the source files. This language was compiled into XML-ish data structures using ply (Python lex/yacc). These structures were parsed into trees using LXML. The trees were combined with templates made in Chameleon. These templates were rendered and previewed with the Bottle web server.
Did it work? Sort of. Source code went in; web pages came out. But it was also complicated and fragile. Moreover, though the automation was there, there wasn’t yet enough abstraction at the source layer. I started thinking about how I could add a source preprocessor.
3.5Enter Racket
I had come across Racket while researching languages suitable for HTML/XML processing. I had unexpectedly learned about the secret kinship of XML and Lisp: though XML is not a programming language, it uses a variant of Lisp syntax. Thus Lisp languages are particularly adept at handling XMLish structures. That was interesting.
After comparing some of the Lisp & Scheme variants, Racket stood out because it had a text-based dialect called Scribble. Scribble could be used to embed code within textual content. That was interesting too. Among other things, this meant Scribble could be used as a general-purpose preprocessor. So I thought I’d see if I could add it to Pollen.
It worked. So well, in fact, that I started thinking about whether I could reimplement other parts of Pollen in Racket. Then I started thinking about reimplementing all of it in Racket.
So I did. And here we are.
3.6What is Pollen?
Pollen is a publishing system built on top of Scribble and Racket. So far I’ve optimized Pollen for digital books, because that’s mainly what I use it for. But it can be used for small projects too.
As a publishing system, Pollen includes:
A programming language. The Pollen language is a variant of Scribble, with specific “dialects” tailored to different kinds of source files. You don’t need to use the programming features to do useful work, but they’re available when you need them.
A set of tools & libraries. Pollen targets HTML output. So it includes a variety of tools that cure common HTML annoyances, including a CSS preprocessor.
A development environment. Pollen works with the DrRacket IDE. It also includes a project web server so you can dynamically preview and revise your publication.
Pollen addresses the deficiencies I experienced with other tools:
Yes, we have a native data structure for HTML. Racket represents HTML structures as X-expressions, which are a variant of the standard Racket data structure, called S-expressions. In other words, not only is there a native representation for HTML, everything in the language is represented this way.
Flexible blending of code, presentation, and content. Pollen is a text-based language. So a Pollen source file might have no code at all. But as a dialect of Scribble & Racket, if you want to mix code with content, you can.
No template language. It’s not necessary, because you can use the whole Racket language, and all the usual Racket syntax, in every Pollen file.
Shallow learning curve. You don’t need to do a lot of setup and configuration to start doing useful work with Pollen. Programmers and non-programmers can easily collaborate. Yes, I concede that if you plan to get serious, you’ll need to learn some Racket. I don’t think you’ll regret it.
I created Pollen to overcome limitations & frustrations I repeatedly encountered with existing web-publishing tools.
If you agree with my characterization of those problems, then you’ll probably like the solution that Pollen offers. If not, you probably won’t.
3.1Web development and its discontents
I made my first web page in 1994, shortly after the web was invented. I opened my text editor (at the time, BBEdit), pecked out <html><body>Helloworld</body></html>, then loaded it in Mosaic. So did a million other nerds.
If you weren’t around then, you didn’t miss much. Everything about the web was horrible: the web browsers, the computers running the browsers, the dial-up connections feeding the browsers, and of course HTML itself. At that point, the desktop-software experience was already slick and refined. By comparison, using the web felt like banging rocks together.
That’s no longer true. The web is now 20 years old. During that time, most parts of the web have improved dramatically — for instance, the connections are faster, the browsers are more sophisticated, the screens have more pixels.
But one part has not improved: the way we make web pages. Over the years, tools promising to simplify HTML development have come and mostly gone — from PageMill to Dreamweaver to WordPress to Jekyll. Meanwhile, true web jocks have remained loyal to the original HTML power tool: the humble text editor.
In one way, this makes sense. Web pages are mostly made of text-based data — HTML, CSS, JavaScript, and so on — and the simplest way to mainpulate this data is with a text editor. While HTML and CSS are not programming languages, they lend themselves to semantic and logical structure that’s most easily expressed by editing them as text. Furthermore, text-based editing makes debugging and performance improvements easier.
But text-based editing is also limited. Though the underlying description of a web page is notionally human-readable, it’s largely optimized to be readable by other software — namely, web browsers. HTML markup in particular is verbose and easily mistyped. And isn’t it fatally dull to manage all the boilerplate, like surrounding every paragraph with <p>...</p>? Yes, it is.
For these reasons, much of web development should lend itself to abstraction & automation. Abstraction means consolidating repetitve, complex patterns into simpler, parameterized forms. Automation means avoiding the manual drudgery of generating the output files. But in practice, tools that enable this abstraction & automation have been slow to arrive, and most have come hobbled with unacceptable deficiencies.
3.2The better idea: a programming model
Parallel with my HTML education, I also goofed around with various programming languages — C, C++, Perl, Java, PHP, JavaScript, Python. Unlike HTML, programming languages excel at abstraction and automation. This seemed like the obvious direction for web development to go.
What distinguishes the text-editing model from the programming model? It’s a matter of direct vs. indirect manipulation of output. The text-editing model treats HTML as something to be written directly with a text editor. Whereas the programming model treats HTML — or whatever the output is — as the result of compiling a set of source files, which are written in a programming language. The costs of working indirectly via the programming language are offset by the benefits of abstraction & automation.
On the early web, the text-editing model was appealingly precise and quick. On small projects, it worked well enough. But as projects grew, the text-editing model was going to lose steam. I wasn’t the only one to notice. Shortly after those million nerds made their first web page by hand, many of them set about devising ways to apply a programming model to web development.
3.3“Now you have two problems”
What followed was a steady stream of products, frameworks, tools, and content management systems that claimed to bring a programming model to web development. Some were better than others. But none of them displaced the text editor as the preferred tool of web developers.
Why not? All these tools promised a great leap forward in solving the web-development problem. In practice, they simply redistributed the pain. I needn’t bore you with enumerating the deficiencies of specific tools, because they’ve tended to fail in the same thematic ways:
No native data structure for HTML. Core to any programming model is data structures. Good data structures make processing easy; bad ones make it hard. Even though HTML has a well documented format, rarely has it been handled within a programming system with a native, intuitive data structure. Instead, it’s either been treated as a string (wrong), a tree (also wrong), or some magical parsed object. This has made working with HTML in programming environments needlessly difficult.
Mandatory separation of code, presentation, and content. This principle has often been held out as an ideal in web development. But it’s also counterintuitive, because an HTML page naturally contains all three. If you want to separate them, your tools should let you. But if you don’t, your tools shouldn’t make you.
Compromised template languages. Seems like every programming language has at least 10 templating systems for HTML, all of which require you to learn a new “template language” that offers the worst of both worlds: fewer features and different syntax than the underlying language.
Steep learning curves. Web programmers have often chided designers for not knowing how to code. But programming-based web-development tools have often had a high initial learning curve that requires you to throw out your existing workflow. Programmers built these tools — no surprise that programmers have been more comfortable with them.
I’ve tried a lot of these tools over the years. Some I liked. Some I didn’t. Invariably, however, whenever I could still make do with hand-editing an HTML project, I would. After trying to cajole the web framework du jour into doing my bidding, it was relaxing to trade off some efficiency for control.
3.4Rethinking the solution for digital books
In 2008, I launched a website called Typography for Lawyers. Initially, I’d conceived of it as a book. Then I thought “no one’s going to publish that.” So it became a website, that I aimed to make as book-like as possible. But hand-editing wasn’t going to be enough.
So I used WordPress. The major chore became scraping out all the crap that typically lives in blog templates. Largely because of this, people liked the site, because it was simpler & cleaner than the usual WordPress website.
Eventually, a publisher offered to release it as a paperback. Later came the inevitable request to make it into a Kindle book. As a fan of typography, I hate the Kindle. The layout controls are coarse, and so is the reading experience. But I didn’t run and hide. Basically a Kindle book is a little website made with 1995-era HTML. So I coded up some tools in Perl to convert my book to Kindle format while preserving the formatting and images as well as possible.
At that point, I noticed I had converted Typography for Lawyers into web format twice, using two different sets of tools. Before someone asked me to do it a third time, I started thinking about how I might create source code for the book that allowed me to render it into different formats.
This was the beginning of the Pollen project.
I wrote the initial version of Pollen in Python. I devised a simplified markup-notation language for the source files. This language was compiled into XML-ish data structures using ply (Python lex/yacc). These structures were parsed into trees using LXML. The trees were combined with templates made in Chameleon. These templates were rendered and previewed with the Bottle web server.
Did it work? Sort of. Source code went in; web pages came out. But it was also complicated and fragile. Moreover, though the automation was there, there wasn’t yet enough abstraction at the source layer. I started thinking about how I could add a source preprocessor.
3.5Enter Racket
I had come across Racket while researching languages suitable for HTML/XML processing. I had unexpectedly learned about the secret kinship of XML and Lisp: though XML is not a programming language, it uses a variant of Lisp syntax. Thus Lisp languages are particularly adept at handling XMLish structures. That was interesting.
After comparing some of the Lisp & Scheme variants, Racket stood out because it had a text-based dialect called Scribble. Scribble could be used to embed code within textual content. That was interesting too. Among other things, this meant Scribble could be used as a general-purpose preprocessor. So I thought I’d see if I could add it to Pollen.
It worked. So well, in fact, that I started thinking about whether I could reimplement other parts of Pollen in Racket. Then I started thinking about reimplementing all of it in Racket.
So I did. And here we are.
3.6What is Pollen?
Pollen is a publishing system built on top of Scribble and Racket. So far I’ve optimized Pollen for digital books, because that’s mainly what I use it for. But it can be used for small projects too.
As a publishing system, Pollen includes:
A programming language. The Pollen language is a variant of Scribble, with specific “dialects” tailored to different kinds of source files. You don’t need to use the programming features to do useful work, but they’re available when you need them.
A set of tools & libraries. Pollen targets HTML output. So it includes a variety of tools that cure common HTML annoyances, including a CSS preprocessor.
A development environment. Pollen works with the DrRacket IDE. It also includes a project web server so you can dynamically preview and revise your publication.
Pollen addresses the deficiencies I experienced with other tools:
Yes, we have a native data structure for HTML. Racket represents HTML structures as X-expressions, which are a variant of the standard Racket data structure, called S-expressions. In other words, not only is there a native representation for HTML, everything in the language is represented this way.
Flexible blending of code, presentation, and content. Pollen is a text-based language. So a Pollen source file might have no code at all. But as a dialect of Scribble & Racket, if you want to mix code with content, you can.
No template language. It’s not necessary, because you can use the whole Racket language, and all the usual Racket syntax, in every Pollen file.
Shallow learning curve. You don’t need to do a lot of setup and configuration to start doing useful work with Pollen. Programmers and non-programmers can easily collaborate. Yes, I concede that if you plan to get serious, you’ll need to learn some Racket. I don’t think you’ll regret it.
\ No newline at end of file
diff --git a/doc/Cache.html b/doc/Cache.html
index d362bdf..f5f15c4 100644
--- a/doc/Cache.html
+++ b/doc/Cache.html
@@ -1,2 +1,2 @@
-10.1 Cache
The slowest part of a render is parsing and decoding the source file. Often, previewing a single source file necessarily means decoding others (for instance templates, or other source files that are linked into the main source file). But usually, only one source file is changing at a time. Therefore, Pollen stores copies of the exports of source files — namely, whatever is stored in doc and metas — in the cache so they can be reused.
A parameter that refers to the current cache. It is initialized with make-cache.
The cache is a hash table that uses the complete path of a source file as its keys. The value associated with each of these keys is a subcache — another hash table with keys 'doc, 'metas (for storing the exports of the source file) and 'mod-time (for storing the modification time, provided by file-or-directory-modify-seconds).
Similar to (dynamic-requiresource-pathkey), except that it first tries to retrieve the requested value out of current-cache. If it’s not there, or out of date, dynamic-require is used to update the value.
The only keys supported are 'doc, 'metas, and 'mod-time.
If you want the speed benefit of the cache, you should always use cached-require to get data from Pollen source files. That doesn’t mean you can’t still use functions like require, local-require, and dynamic-require. They’ll just be slower.
The slowest part of a render is parsing and decoding the source file. Often, previewing a single source file necessarily means decoding others (for instance templates, or other source files that are linked into the main source file). But usually, only one source file is changing at a time. Therefore, Pollen stores copies of the exports of source files — namely, whatever is stored in doc and metas — in the cache so they can be reused.
A parameter that refers to the current cache. It is initialized with make-cache.
The cache is a hash table that uses the complete path of a source file as its keys. The value associated with each of these keys is a subcache — another hash table with keys 'doc, 'metas (for storing the exports of the source file) and 'mod-time (for storing the modification time, provided by file-or-directory-modify-seconds).
Similar to (dynamic-requiresource-pathkey), except that it first tries to retrieve the requested value out of current-cache. If it’s not there, or out of date, dynamic-require is used to update the value.
The only keys supported are 'doc, 'metas, and 'mod-time.
If you want the speed benefit of the cache, you should always use cached-require to get data from Pollen source files. That doesn’t mean you can’t still use functions like require, local-require, and dynamic-require. They’ll just be slower.
\ No newline at end of file
diff --git a/doc/Decode.html b/doc/Decode.html
index a9edc39..48dc1fb 100644
--- a/doc/Decode.html
+++ b/doc/Decode.html
@@ -1,2 +1,2 @@
-10.2 Decode
The doc export of a Pollen markup file is a simple X-expression. Decoding refers to any post-processing of this X-expression. The pollen/decode module provides tools for creating decoders.
The decode step can happen separately from the compilation of the file. But you can also attach a decoder to the markup file’s root node, so the decoding happens automatically when the markup is compiled, and thus automatically incorporated into doc. (Following this approach, you could also attach multiple decoders to different tags within doc.)
You can, of course, embed function calls within Pollen markup. But since markup is optimized for authors, decoding is useful for operations that can or should be moved out of the authoring layer.
One example is presentation and layout. For instance, detect-paragraphs is a decoder function that lets authors mark paragraphs in their source simply by using two carriage returns.
Another example is conversion of output into a particular data format. Most Pollen functions are optimized for HTML output, but one could write a decoder that targets another format.
Recursively process a tagged-xexpr, usually the one exported from a Pollen source file as doc.
This function doesn’t do much on its own. Rather, it provides the hooks upon which harder-working functions can be hung.
Recall from [future link: Pollen mechanics] that any tag can have a function attached to it. By default, the tagged-xexpr from a source file is tagged with root. So the typical way to use decode is to attach your decoding functions to it, and then define root to invoke your decode function. Then it will be automatically applied to every doc during compile.
For instance, here’s how decode is attached to root in Butterick’s Practical Typography. There’s not much to it —
The hyphenate function is not part of Pollen, but rather the hyphenate package, which you can install separately.
This illustrates another important point: even though decode presents an imposing list of arguments, you’re unlikely to use all of them at once. These represent possibilities, not requirements. For instance, let’s see what happens when decode is invoked without any of its optional arguments.
Right — nothing. That’s because the default value for the decoding arguments is the identity function, (λ(x)x). So all the input gets passed through intact unless another action is specified.
The *-proc arguments of decode take procedures that are applied to specific categories of elements within txexpr.
The txexpr-tag-proc argument is a procedure that handles X-expression tags.
Examples:
> (definetx'(p"I'm from a strange"(strong"namespace")))
;Tags are symbols, so a tag-proc should return a symbol
'(ns:p "I'm from a strange" (ns:strong "namespace"))
The txexpr-attrs-proc argument is a procedure that handles lists of X-expression attributes. (The txexpr module, included at no extra charge with Pollen, includes useful helper functions for dealing with these attribute lists.)
Examples:
> (definetx'(p[[id"first"]]"If I only had a brain."))
'(p ((class "PhD") (id "first")) "If I only had a brain.")
Note that txexpr-attrs-proc will change the attributes of every tagged X-expression, even those that don’t have attributes. This is useful, because sometimes you want to add attributes where none existed before. But be careful, because the behavior may make your processing function overinclusive.
Examples:
> (definetx'(div(p[[id"first"]]"If I only had a brain.")
'(div (p ((class "PhD") (id "first")) "If I only had a brain.") (p "Me too."))
The txexpr-elements-proc argument is a procedure that operates on the list of elements that represents the content of each tagged X-expression. Note that each element of an X-expression is subject to two passes through the decoder: once now, as a member of the list of elements, and also later, through its type-specific decoder (i.e., string-proc, symbol-proc, and so on).
So why do you need txexpr-elements-proc? Because some types of element decoding depend on context, thus it’s necessary to handle the elements as a group. For instance, the doubling function above, though useless, requires handling the element list as a whole, because elements are being added.
A more useful example: paragraph detection. The behavior is not merely a map across each element:
;Context matters. Trailing whitespace is ignored ...
> (paras'(body"The first paragraph.""\n\n"))
'(body "The first paragraph.")
;... but whitespace between strings is converted to a break.
> (paras'(body"The first paragraph.""\n\n""And another."))
'(body (p "The first paragraph.") (p "And another."))
;A combination of both types
> (paras'(body"The first paragraph.""\n\n""And another.""\n\n"))
'(body (p "The first paragraph.") (p "And another."))
The block-txexpr-proc argument and the inline-txexpr-proc arguments are procedures that operate on tagged X-expressions. If the X-expression meets the block-txexpr? test, it is processed by block-txexpr-proc. Otherwise, it is processed by inline-txexpr-proc. Thus every tagged X-expression will be handled by one or the other. Of course, if you want block and inline elements to be handled the same way, you can set block-txexpr-proc and inline-txexpr-proc to be the same procedure.
Examples:
> (definetx'(div"Please"(em"mind the gap")(h1"Tuesdays only")))
'(ns:div "Please" (ns:em "mind the gap") (ns:h1 "Tuesdays only"))
The string-proc, symbol-proc, valid-char-proc, and cdata-proc arguments are procedures that operate on X-expressions that are strings, symbols, valid-chars, and CDATA, respectively. Deliberately, the output contracts for these procedures accept any kind of X-expression (meaning, the procedure can change the X-expression type).
Examples:
;A div with string, entity, character, and cdata elements
Finally, the tags-to-exclude argument is a list of tags that will be exempted from decoding. Though you could get the same result by testing the input within the individual decoding functions, that’s tedious and potentially slower.
Examples:
> (definetx'(p"I really think"(em"italics")"should be lowercase."))
'(p "I REALLY THINK" (em "italics") "SHOULD BE LOWERCASE.")
The tags-to-exclude argument is useful if you’re decoding source that’s destined to become HTML. According to the HTML spec, material within a <style> or <script> block needs to be preserved literally. In this example, if the CSS and JavaScript blocks are capitalized, they won’t work. So exclude '(stylescript), and problem solved.
(script ((type "text/javascript")) "var area = h * w;"))
10.2.1Block
Because it’s convenient, Pollen categorizes tagged X-expressions into two categories: block and inline. Why is it convenient? When using decode, you often want to treat the two categories differently. Not that you have to. But this is how you can.
Adds a tag to project-block-tags so that block-txexpr? will report it as a block, and decode will process it with block-txexpr-proc rather than inline-txexpr-proc.
Pollen tries to do the right thing without being told. But this is the rare case where you have to be explicit. If you introduce a tag into your markup that you want treated as a block, you must use this function to identify it, or you will get spooky behavior later on.
For instance, detect-paragraphs knows that block elements in the markup shouldn’t be wrapped in a p tag. So if you introduce a new block element called bloq without registering it as a block, misbehavior will follow:
> (paras'(body"I want to be a paragraph.""\n\n"(bloq"But not me.")))
'(body (p "I want to be a paragraph.") (bloq "But not me."))
;Right: bloq is treated as a block
If you find the idea of registering block tags unbearable, good news. The project-block-tags include the standard HTML block tags by default. So if you just want to use things like div and p and h1–h6, you’ll get the right behavior for free.
Predicate that tests whether v is a tagged X-expression, and if so, whether the tag is among the project-block-tags. If not, it is treated as inline. To adjust how this test works, use register-block-tag.
10.2.2Typography
An assortment of typography & layout functions, designed to be used with decode. These aren’t hard to write. So if you like these, use them. If not, make your own.
A predicate that returns #t for any stringlike v that’s entirely whitespace, but also the empty string, as well as lists and vectors that are made only of whitespace? members. Following the regexp-match convention, whitespace? does not return #t for a nonbreaking space. If you prefer that behavior, use whitespace/nbsp?.
Within tagged-xexpr-elements, convert occurrences of linebreak-sep ("\n" by default) to linebreak, but only if linebreak-sep does not occur between blocks (see block-txexpr?). Why? Because block-level elements automatically display on a new line, so adding linebreak would be superfluous. In that case, linebreak-sep just disappears.
Find paragraphs within elements, as denoted by paragraph-sep, and wrap them with paragraph-tag, unless the element is already a block-txexpr? (because in that case, the wrapping is superfluous). Thus, as a consequence, if paragraph-sep occurs between two blocks, it’s ignored.
The paragraph-tag argument sets the tag used to wrap paragraphs.
The linebreak-proc argument allows you to use a different linebreaking procedure other than the usual detect-linebreaks.
Find single or double quote marks at the beginning of tx and wrap them in an X-expression with the tag single-preprender or double-preprender, respectively. The default values are 'squo and 'dquo.
In pro typography, quotation marks at the beginning of a line or paragraph are often shifted into the margin slightly to make them appear more optically aligned with the left edge of the text. With a reflowable layout model like HTML, you don’t know where your line breaks will be.
This function will simply insert the 'squo and 'dquo tags, which provide hooks that let you do the actual hanging via CSS, like so (actual measurement can be refined to taste):
squo {margin-left: -0.25em;}
dquo {margin-left: -0.50em;}
Be warned: there are many edge cases this function does not handle well.
The doc export of a Pollen markup file is a simple X-expression. Decoding refers to any post-processing of this X-expression. The pollen/decode module provides tools for creating decoders.
The decode step can happen separately from the compilation of the file. But you can also attach a decoder to the markup file’s root node, so the decoding happens automatically when the markup is compiled, and thus automatically incorporated into doc. (Following this approach, you could also attach multiple decoders to different tags within doc.)
You can, of course, embed function calls within Pollen markup. But since markup is optimized for authors, decoding is useful for operations that can or should be moved out of the authoring layer.
One example is presentation and layout. For instance, detect-paragraphs is a decoder function that lets authors mark paragraphs in their source simply by using two carriage returns.
Another example is conversion of output into a particular data format. Most Pollen functions are optimized for HTML output, but one could write a decoder that targets another format.
Recursively process a tagged-xexpr, usually the one exported from a Pollen source file as doc.
This function doesn’t do much on its own. Rather, it provides the hooks upon which harder-working functions can be hung.
Recall from [future link: Pollen mechanics] that any tag can have a function attached to it. By default, the tagged-xexpr from a source file is tagged with root. So the typical way to use decode is to attach your decoding functions to it, and then define root to invoke your decode function. Then it will be automatically applied to every doc during compile.
For instance, here’s how decode is attached to root in Butterick’s Practical Typography. There’s not much to it —
The hyphenate function is not part of Pollen, but rather the hyphenate package, which you can install separately.
This illustrates another important point: even though decode presents an imposing list of arguments, you’re unlikely to use all of them at once. These represent possibilities, not requirements. For instance, let’s see what happens when decode is invoked without any of its optional arguments.
Right — nothing. That’s because the default value for the decoding arguments is the identity function, (λ(x)x). So all the input gets passed through intact unless another action is specified.
The *-proc arguments of decode take procedures that are applied to specific categories of elements within txexpr.
The txexpr-tag-proc argument is a procedure that handles X-expression tags.
Examples:
> (definetx'(p"I'm from a strange"(strong"namespace")))
;Tags are symbols, so a tag-proc should return a symbol
'(ns:p "I'm from a strange" (ns:strong "namespace"))
The txexpr-attrs-proc argument is a procedure that handles lists of X-expression attributes. (The txexpr module, included at no extra charge with Pollen, includes useful helper functions for dealing with these attribute lists.)
Examples:
> (definetx'(p[[id"first"]]"If I only had a brain."))
'(p ((class "PhD") (id "first")) "If I only had a brain.")
Note that txexpr-attrs-proc will change the attributes of every tagged X-expression, even those that don’t have attributes. This is useful, because sometimes you want to add attributes where none existed before. But be careful, because the behavior may make your processing function overinclusive.
Examples:
> (definetx'(div(p[[id"first"]]"If I only had a brain.")
'(div (p ((class "PhD") (id "first")) "If I only had a brain.") (p "Me too."))
The txexpr-elements-proc argument is a procedure that operates on the list of elements that represents the content of each tagged X-expression. Note that each element of an X-expression is subject to two passes through the decoder: once now, as a member of the list of elements, and also later, through its type-specific decoder (i.e., string-proc, symbol-proc, and so on).
So why do you need txexpr-elements-proc? Because some types of element decoding depend on context, thus it’s necessary to handle the elements as a group. For instance, the doubling function above, though useless, requires handling the element list as a whole, because elements are being added.
A more useful example: paragraph detection. The behavior is not merely a map across each element:
;Context matters. Trailing whitespace is ignored ...
> (paras'(body"The first paragraph.""\n\n"))
'(body "The first paragraph.")
;... but whitespace between strings is converted to a break.
> (paras'(body"The first paragraph.""\n\n""And another."))
'(body (p "The first paragraph.") (p "And another."))
;A combination of both types
> (paras'(body"The first paragraph.""\n\n""And another.""\n\n"))
'(body (p "The first paragraph.") (p "And another."))
The block-txexpr-proc argument and the inline-txexpr-proc arguments are procedures that operate on tagged X-expressions. If the X-expression meets the block-txexpr? test, it is processed by block-txexpr-proc. Otherwise, it is processed by inline-txexpr-proc. Thus every tagged X-expression will be handled by one or the other. Of course, if you want block and inline elements to be handled the same way, you can set block-txexpr-proc and inline-txexpr-proc to be the same procedure.
Examples:
> (definetx'(div"Please"(em"mind the gap")(h1"Tuesdays only")))
'(ns:div "Please" (ns:em "mind the gap") (ns:h1 "Tuesdays only"))
The string-proc, symbol-proc, valid-char-proc, and cdata-proc arguments are procedures that operate on X-expressions that are strings, symbols, valid-chars, and CDATA, respectively. Deliberately, the output contracts for these procedures accept any kind of X-expression (meaning, the procedure can change the X-expression type).
Examples:
;A div with string, entity, character, and cdata elements
Finally, the tags-to-exclude argument is a list of tags that will be exempted from decoding. Though you could get the same result by testing the input within the individual decoding functions, that’s tedious and potentially slower.
Examples:
> (definetx'(p"I really think"(em"italics")"should be lowercase."))
'(p "I REALLY THINK" (em "italics") "SHOULD BE LOWERCASE.")
The tags-to-exclude argument is useful if you’re decoding source that’s destined to become HTML. According to the HTML spec, material within a <style> or <script> block needs to be preserved literally. In this example, if the CSS and JavaScript blocks are capitalized, they won’t work. So exclude '(stylescript), and problem solved.
(script ((type "text/javascript")) "var area = h * w;"))
10.2.1Block
Because it’s convenient, Pollen categorizes tagged X-expressions into two categories: block and inline. Why is it convenient? When using decode, you often want to treat the two categories differently. Not that you have to. But this is how you can.
Adds a tag to project-block-tags so that block-txexpr? will report it as a block, and decode will process it with block-txexpr-proc rather than inline-txexpr-proc.
Pollen tries to do the right thing without being told. But this is the rare case where you have to be explicit. If you introduce a tag into your markup that you want treated as a block, you must use this function to identify it, or you will get spooky behavior later on.
For instance, detect-paragraphs knows that block elements in the markup shouldn’t be wrapped in a p tag. So if you introduce a new block element called bloq without registering it as a block, misbehavior will follow:
> (paras'(body"I want to be a paragraph.""\n\n"(bloq"But not me.")))
'(body (p "I want to be a paragraph.") (bloq "But not me."))
;Right: bloq is treated as a block
If you find the idea of registering block tags unbearable, good news. The project-block-tags include the standard HTML block tags by default. So if you just want to use things like div and p and h1–h6, you’ll get the right behavior for free.
Predicate that tests whether v is a tagged X-expression, and if so, whether the tag is among the project-block-tags. If not, it is treated as inline. To adjust how this test works, use register-block-tag.
10.2.2Typography
An assortment of typography & layout functions, designed to be used with decode. These aren’t hard to write. So if you like these, use them. If not, make your own.
A predicate that returns #t for any stringlike v that’s entirely whitespace, but also the empty string, as well as lists and vectors that are made only of whitespace? members. Following the regexp-match convention, whitespace? does not return #t for a nonbreaking space. If you prefer that behavior, use whitespace/nbsp?.
Within tagged-xexpr-elements, convert occurrences of linebreak-sep ("\n" by default) to linebreak, but only if linebreak-sep does not occur between blocks (see block-txexpr?). Why? Because block-level elements automatically display on a new line, so adding linebreak would be superfluous. In that case, linebreak-sep just disappears.
Find paragraphs within elements (as denoted by paragraph-sep) and wrap them with paragraph-tag. Also handle linebreaks using detect-linebreaks.
If element is already a block-txexpr?, it will not be wrapped as a paragraph (because in that case, the wrapping would be superfluous). Thus, as a consequence, if paragraph-sep occurs between two blocks, it will be ignored (as in the example below using two sequential 'div blocks.)
The paragraph-tag argument sets the tag used to wrap paragraphs.
The linebreak-proc argument allows you to use a different linebreaking procedure other than the usual detect-linebreaks.
Find single or double quote marks at the beginning of tx and wrap them in an X-expression with the tag single-preprender or double-preprender, respectively. The default values are 'squo and 'dquo.
In pro typography, quotation marks at the beginning of a line or paragraph are often shifted into the margin slightly to make them appear more optically aligned with the left edge of the text. With a reflowable layout model like HTML, you don’t know where your line breaks will be.
This function will simply insert the 'squo and 'dquo tags, which provide hooks that let you do the actual hanging via CSS, like so (actual measurement can be refined to taste):
squo {margin-left: -0.25em;}
dquo {margin-left: -0.50em;}
Be warned: there are many edge cases this function does not handle well.
\ No newline at end of file
diff --git a/doc/File_formats.html b/doc/File_formats.html
index 27d3ec3..b933cac 100644
--- a/doc/File_formats.html
+++ b/doc/File_formats.html
@@ -1,2 +1,2 @@
-8 File formats
The Pollen language is divided into variants, or dialects, that are tailored to suit each of the core source formats.
These dialects can be invoked one of two ways: either by invoking a specific dialect in the first line of the file (also known as the #lang line), or by using the generic #lang pollen as the first line, and then the correct dialect will be automatically selected based on the source file extension.
If the #lang line specifies a dialect different from the one specified by the file extension, the #lang line will take precedence.
For ease of use, the behavior of the Pollen language departs from the standard Racket language in several ways. The differences are noted below.
8.1.1Command syntax using ◊
Commands must start with the special lozenge character ◊. Other material is interpreted as plain text. See ◊ command overview for more.
How is this different from Racket? In Racket, everything is a command, and plain text must be quoted.
8.1.2Any command is valid
There are no undefined commands in Pollen. If a command has not already been defined, it’s treated as a tag function. See ◊ command overview for more.
How is this different from Racket? In Racket, if you try to treat an identifier as a function before defining it with define, you’ll get an error.
8.1.3Standard exports
By default, every Pollen source file exports two symbols, which you can access by using the source file with require:
'doc contains the output of the file. The type of output depends on the source format (documented below).
'metas is a hash of key–value pairs with extra information that is extracted from the source. These 'metas will always contain the key 'here-path, which returns a string representation of the full path to the source file. Beyond that, the only 'metas are the ones that are specified within the source file (see the source formats below for more detail on how to specify metas).
How is this different from Racket? In Racket, you must explicitly define and then provide any values you want to export.
8.1.4Custom exports
Any value or function that is defined within the source file using define is automatically exported.
How is this different from Racket? In Racket, you must explicitly provide any values you want to export. Unlike Racket, every Pollen source file impliedly uses (provide(all-defined-out)).
8.1.5The project-require.rkt file
If a file called project-require.rkt exists in the same directory with a source file, it’s automatically imported when the source file is compiled.
How is this different from Racket? In Racket, you must explicitly import files using require.
8.1.6Preprocessor (.pp extension)
Invoke the preprocessor dialect by using #langpollen/pre as the first line of your source file, or by using #langpollen with a file extension of .pp. These forms are equivalent:
When no dialect is explicitly specified by either the #lang line or the file extension, Pollen will default to using the preprocessor dialect. For instance, this file will be treated as preprocessor source:
Of course, you’re better off specifying the preprocessor dialect explicitly rather than relying on this default behavior.
The output of the preprocessor dialect, provided by 'doc, is plain text.
8.1.7Markdown (.pmd extension)
Invoke the Markdown dialect by using #langpollen/markdown as the first line of your source file, or by using #langpollen with a file extension of .pmd. These forms are equivalent:
The output of the Markdown dialect, provided by 'doc, is a tagged X-expression.
8.1.8Markup (.pm extension)
Invoke the Pollen markup dialect by using #langpollen/markup as the first line of your source file, or by using #langpollen with a file extension of .pm. These forms are equivalent:
The output of the Pollen markup dialect, provided by 'doc, is a tagged X-expression.
8.1.9Pagetree (.ptree extension)
Invoke the pagetree dialect by using #langpollen/ptree as the first line of your source file, or by using #langpollen with a file extension of .ptree. These forms are equivalent:
The output of the pagetree dialect, provided by 'doc, is a pagetree? that is checked for correctness using validate-pagetree.
8.2Utility formats
These aren’t source formats because they don’t contain a #lang pollen line. But for convenience, they get special handling by the Pollen project server.
8.2.1Scribble (.scrbl extension)
Scribble files are recognized by the project server and can be compiled and previewed in single-page mode.
8.2.2Null (.p extension)
Files with the null extension are simply rendered as a copy of the file without the extension, so index.html.p becomes index.html.
This can be useful you’re managing your project with git. Most likely you’ll want to ignore *.html and other file types that are frequently regenerated by the project server. But if you have isolated static files — for instance, a index.html that doesn’t have source associated with it — they’ll be ignored too. You can cure this problem by appending the null extension to these static files, so they’ll be tracked in your source system without actually being source files.
The Pollen language is divided into variants, or dialects, that are tailored to suit each of the core source formats.
These dialects can be invoked one of two ways: either by invoking a specific dialect in the first line of the file (also known as the #lang line), or by using the generic #lang pollen as the first line, and then the correct dialect will be automatically selected based on the source file extension.
If the #lang line specifies a dialect different from the one specified by the file extension, the #lang line will take precedence.
For ease of use, the behavior of the Pollen language departs from the standard Racket language in several ways. The differences are noted below.
8.1.1Command syntax using ◊
Commands must start with the special lozenge character ◊. Other material is interpreted as plain text. See ◊ command overview for more.
How is this different from Racket? In Racket, everything is a command, and plain text must be quoted.
8.1.2Any command is valid
There are no undefined commands in Pollen. If a command has not already been defined, it’s treated as a tag function. See ◊ command overview for more.
How is this different from Racket? In Racket, if you try to treat an identifier as a function before defining it with define, you’ll get an error.
8.1.3Standard exports
By default, every Pollen source file exports two symbols, which you can access by using the source file with require:
'doc contains the output of the file. The type of output depends on the source format (documented below).
'metas is a hash of key–value pairs with extra information that is extracted from the source. These 'metas will always contain the key 'here-path, which returns a string representation of the full path to the source file. Beyond that, the only 'metas are the ones that are specified within the source file (see the source formats below for more detail on how to specify metas).
How is this different from Racket? In Racket, you must explicitly define and then provide any values you want to export.
8.1.4Custom exports
Any value or function that is defined within the source file using define is automatically exported.
How is this different from Racket? In Racket, you must explicitly provide any values you want to export. Unlike Racket, every Pollen source file impliedly uses (provide(all-defined-out)).
8.1.5The project-require.rkt file
If a file called project-require.rkt exists in the same directory with a source file, it’s automatically imported when the source file is compiled.
How is this different from Racket? In Racket, you must explicitly import files using require.
8.1.6Preprocessor (.pp extension)
Invoke the preprocessor dialect by using #langpollen/pre as the first line of your source file, or by using #langpollen with a file extension of .pp. These forms are equivalent:
When no dialect is explicitly specified by either the #lang line or the file extension, Pollen will default to using the preprocessor dialect. For instance, this file will be treated as preprocessor source:
Of course, you’re better off specifying the preprocessor dialect explicitly rather than relying on this default behavior.
The output of the preprocessor dialect, provided by 'doc, is plain text.
8.1.7Markdown (.pmd extension)
Invoke the Markdown dialect by using #langpollen/markdown as the first line of your source file, or by using #langpollen with a file extension of .pmd. These forms are equivalent:
The output of the Markdown dialect, provided by 'doc, is a tagged X-expression.
8.1.8Markup (.pm extension)
Invoke the Pollen markup dialect by using #langpollen/markup as the first line of your source file, or by using #langpollen with a file extension of .pm. These forms are equivalent:
The output of the Pollen markup dialect, provided by 'doc, is a tagged X-expression.
8.1.9Pagetree (.ptree extension)
Invoke the pagetree dialect by using #langpollen/ptree as the first line of your source file, or by using #langpollen with a file extension of .ptree. These forms are equivalent:
The output of the pagetree dialect, provided by 'doc, is a pagetree? that is checked for correctness using validate-pagetree.
8.2Utility formats
These aren’t source formats because they don’t contain a #lang pollen line. But for convenience, they get special handling by the Pollen project server.
8.2.1Scribble (.scrbl extension)
Scribble files are recognized by the project server and can be compiled and previewed in single-page mode.
8.2.2Null (.p extension)
Files with the null extension are simply rendered as a copy of the file without the extension, so index.html.p becomes index.html.
This can be useful you’re managing your project with git. Most likely you’ll want to ignore *.html and other file types that are frequently regenerated by the project server. But if you have isolated static files — for instance, a index.html that doesn’t have source associated with it — they’ll be ignored too. You can cure this problem by appending the null extension to these static files, so they’ll be tracked in your source system without actually being source files.
\ No newline at end of file
diff --git a/doc/Installation.html b/doc/Installation.html
index 4ec92ab..d8a713d 100644
--- a/doc/Installation.html
+++ b/doc/Installation.html
@@ -1,4 +1,4 @@
-1 Installation
\ No newline at end of file
diff --git a/doc/Module_reference.html b/doc/Module_reference.html
index 515cd5b..9f78e6a 100644
--- a/doc/Module_reference.html
+++ b/doc/Module_reference.html
@@ -1,2 +1,2 @@
-10 Module reference
\ No newline at end of file
diff --git a/doc/Pagetree.html b/doc/Pagetree.html
index 777ef63..c82f27d 100644
--- a/doc/Pagetree.html
+++ b/doc/Pagetree.html
@@ -1,2 +1,2 @@
-10.4 Pagetree
A pagetree is a hierarchical list of Pollen output files. A pagetree source file has the extension .ptree. A pagetree provides a convenient way of separating the structure of the pages from the page sources, and navigating around this structure.
Pagetrees are made of pagenodes. Usually these pagenodes will be names of output files in your project. (If you think it would’ve been more logical to just call them “pages,” perhaps. When I think of a web page, I think of a file on a disk. Whereas pagenodes may — and often do — refer to files that don’t yet exist.)
Books and other long documents are usually organized in a structured way — at minimum they have a sequence of pages, but more often they have sections with subsequences within. Individual Pollen source files don’t know anything about how they’re connected to other files. In theory, you could maintain this information within each source file. This would be a poor use of human energy. Let the pagetree figure it out.
Test whether possible-pagetree is a valid pagetree. It must be a txexpr? where all elements are pagenode?, and each is unique within possible-pagetree (not counting the root node).
Test whether possible-pagenode is a valid pagenode. A pagenode can be any symbol? that is not whitespace/nbsp? Every leaf of a pagetree is a pagenode. In practice, your pagenodes will likely be names of output files.
Pagenodes are symbols (rather than strings) so that pagetrees will be valid tagged X-expressions, which is a more convenient format for validation & processing.
A parameter that defines the default pagetree used by pagetree navigation functions (e.g., parent-pagenode, chidren, et al.) if another is not explicitly specified. Initialized to #f.
Return the pagenode immediately before p. For previous*, return all the pagenodes before p, in sequence. In both cases, return #f if there aren’t any pagenodes. The root pagenode is ignored.
Return the pagenode immediately after p. For next*, return all the pagenodes after p, in sequence. In both cases, return #f if there aren’t any pagenodes. The root pagenode is ignored.
Convert path p to a pagenode — meaning, make it relative to current-project-root, run it through ->output-path, and convert it to a symbol. Does not tell you whether the resultant pagenode actually exists in the current pagetree (for that, use in-pagetree?).
A pagetree is a hierarchical list of Pollen output files. A pagetree source file has the extension .ptree. A pagetree provides a convenient way of separating the structure of the pages from the page sources, and navigating around this structure.
Pagetrees are made of pagenodes. Usually these pagenodes will be names of output files in your project. (If you think it would’ve been more logical to just call them “pages,” perhaps. When I think of a web page, I think of a file on a disk. Whereas pagenodes may — and often do — refer to files that don’t yet exist.)
Books and other long documents are usually organized in a structured way — at minimum they have a sequence of pages, but more often they have sections with subsequences within. Individual Pollen source files don’t know anything about how they’re connected to other files. In theory, you could maintain this information within each source file. This would be a poor use of human energy. Let the pagetree figure it out.
Test whether possible-pagetree is a valid pagetree. It must be a txexpr? where all elements are pagenode?, and each is unique within possible-pagetree (not counting the root node).
Test whether possible-pagenode is a valid pagenode. A pagenode can be any symbol? that is not whitespace/nbsp? Every leaf of a pagetree is a pagenode. In practice, your pagenodes will likely be names of output files.
Pagenodes are symbols (rather than strings) so that pagetrees will be valid tagged X-expressions, which is a more convenient format for validation & processing.
A parameter that defines the default pagetree used by pagetree navigation functions (e.g., parent-pagenode, chidren, et al.) if another is not explicitly specified. Initialized to #f.
Return the pagenode immediately before p. For previous*, return all the pagenodes before p, in sequence. In both cases, return #f if there aren’t any pagenodes. The root pagenode is ignored.
Return the pagenode immediately after p. For next*, return all the pagenodes after p, in sequence. In both cases, return #f if there aren’t any pagenodes. The root pagenode is ignored.
Convert path p to a pagenode — meaning, make it relative to current-project-root, run it through ->output-path, and convert it to a symbol. Does not tell you whether the resultant pagenode actually exists in the current pagetree (for that, use in-pagetree?).
\ No newline at end of file
diff --git a/doc/Render.html b/doc/Render.html
index fb2133b..b816920 100644
--- a/doc/Render.html
+++ b/doc/Render.html
@@ -1,4 +1,4 @@
-10.5 Render
Renders source-path. The rendering behavior depends on the type of source file:
A [pollen/pre] file is rendered without a template.
A [pollen/markup] or [pollen/markdown] file is rendered with a template. If no template is provided with template-path, Pollen finds one using get-template-for.
Be aware that rendering with a template uses include-template within eval. For complex pages, it can be slow the first time. Caching is used to make subsequent requests faster.
For those panicked at the use of eval, please don’t be. As the author of include-template has already advised, “If you insist on dynamicism” — and yes, I do insist — “there is always eval.”
Like render, but saves the file to output-path, overwriting whatever was already there. If no output-path is provided, it’s derived from source-path using ->output-path.
Render multiple source-paths in one go. This can be faster than (for-eachrendersource-paths) if your source-paths rely on a common set of templates. Templates may have their own source files that need to be compiled. If you use render, the templates will be repeatedly (and needlessly) re-compiled. Whereas if you use render-batch, each template will only be compiled once.
Renders source-path. The rendering behavior depends on the type of source file:
A [pollen/pre] file is rendered without a template.
A [pollen/markup] or [pollen/markdown] file is rendered with a template. If no template is provided with template-path, Pollen finds one using get-template-for.
Be aware that rendering with a template uses include-template within eval. For complex pages, it can be slow the first time. Caching is used to make subsequent requests faster.
For those panicked at the use of eval, please don’t be. As the author of include-template has already advised, “If you insist on dynamicism” — and yes, I do insist — “there is always eval.”
Like render, but saves the file to output-path, overwriting whatever was already there. If no output-path is provided, it’s derived from source-path using ->output-path.
Render multiple source-paths in one go. This can be faster than (for-eachrendersource-paths) if your source-paths rely on a common set of templates. Templates may have their own source files that need to be compiled. If you use render, the templates will be repeatedly (and needlessly) re-compiled. Whereas if you use render-batch, each template will only be compiled once.
Find a template file for source-path, with the following priority:
If the metas for source-path have a key for template, then use the value of this key.
If this key doesn’t exist, or if it points to a nonexistent file, look for a default template in the project directory with the name template.[outputextension]. Meaning, if source-path is intro.html.pm, the output path would be intro.html, so the default template would be template.html.
If this file doesn’t exist, use the fallback template as a last resort.
This function is called when a template is needed, but a template-path argument is missing (for instance, in render or render-to-file).
\ No newline at end of file
diff --git a/doc/Tag.html b/doc/Tag.html
index c7c7663..84e77c5 100644
--- a/doc/Tag.html
+++ b/doc/Tag.html
@@ -1,2 +1,2 @@
-10.7 Tag
Make a tag function for id. As arguments, a tag function takes an optional set of X-expression attributes (txexpr-attrs?) followed by X-expression elements (txexpr-elements?). From these, the tag function creates a tagged X-expression using id as the tag.
Entering attributes this way can be cumbersome. So for convenience, a tag function provides an alternative: any symbol + string pairs at the front of your expression will be interpreted as attributes, if the symbols are followed by a colon. If you leave out the colon, the symbols will be interpreted as part of the content of the tag.
Make a tag function for id. As arguments, a tag function takes an optional set of X-expression attributes (txexpr-attrs?) followed by X-expression elements (txexpr-elements?). From these, the tag function creates a tagged X-expression using id as the tag.
Entering attributes this way can be cumbersome. So for convenience, a tag function provides an alternative: any symbol + string pairs at the front of your expression will be interpreted as attributes, if the symbols are followed by a colon. If you leave out the colon, the symbols will be interpreted as part of the content of the tag.
\ No newline at end of file
diff --git a/doc/Template.html b/doc/Template.html
index ae71f18..aaec3da 100644
--- a/doc/Template.html
+++ b/doc/Template.html
@@ -1,2 +1,2 @@
-10.6 Template
Convert xexpr to an HTML string. Similar to xexpr->string, but consistent with the HTML spec, text that appears within script or style blocks will not be escaped.
"<root><script>3 > 2</script>Why is 3 > 2?</root>"
> (->htmltx)
"<root><script>3 > 2</script>Why is 3 > 2?</root>"
The optional keyword arguments html-tag and html-attrs let you set the outer tag and attributes for the generated HTML. If xexpr already has an outer tag or attributes, they will be replaced.
Be careful not to pass existing HTML strings into this function, because the angle brackets will be escaped. Fine if that’s what you want, but you probably don’t.
Find matches for key in value-source, first by looking in its metas (using select-from-metas) and then by looking in its doc (using select-from-doc). With select, you get the first result; with select*, you get them all. In both cases, you get #f if there are no matches.
Look up the value of key in meta-source. The meta-source argument can be either a set of metas (i.e., a hash) or a pagenode?, from which metas are pulled. If no value exists for key, you get #f.
Look up the value of key in doc-source. The doc-source argument can be either be a doc (i.e., a txexpr) or a pagenode?, from which doc is pulled. If no value exists for key, you get #f.
Convenience function for templates that’s simpler to use than plain when. If condition is true, then put the text-to-insert into the template at the current location. Within a template file, usually invoked like so:
◊when/block[condition]{The text to insert.}
The inserted text can contain its own nested Pollen commands.
Convert xexpr to an HTML string. Similar to xexpr->string, but consistent with the HTML spec, text that appears within script or style blocks will not be escaped.
"<root><script>3 > 2</script>Why is 3 > 2?</root>"
> (->htmltx)
"<root><script>3 > 2</script>Why is 3 > 2?</root>"
The optional keyword arguments html-tag and html-attrs let you set the outer tag and attributes for the generated HTML. If xexpr already has an outer tag or attributes, they will be replaced.
Be careful not to pass existing HTML strings into this function, because the angle brackets will be escaped. Fine if that’s what you want, but you probably don’t.
Find matches for key in value-source, first by looking in its metas (using select-from-metas) and then by looking in its doc (using select-from-doc). With select, you get the first result; with select*, you get them all. In both cases, you get #f if there are no matches.
Look up the value of key in meta-source. The meta-source argument can be either a set of metas (i.e., a hash) or a pagenode?, from which metas are pulled. If no value exists for key, you get #f.
Look up the value of key in doc-source. The doc-source argument can be either be a doc (i.e., a txexpr) or a pagenode?, from which doc is pulled. If no value exists for key, you get #f.
Convenience function for templates that’s simpler to use than plain when. If condition is true, then put the text-to-insert into the template at the current location. Within a template file, usually invoked like so:
◊when/block[condition]{The text to insert.}
The inserted text can contain its own nested Pollen commands.
\ No newline at end of file
diff --git a/doc/Top.html b/doc/Top.html
index e97c62c..dc15c9c 100644
--- a/doc/Top.html
+++ b/doc/Top.html
@@ -1,2 +1,2 @@
-10.8 Top
You’ll probably never invoke this module directly. But it’s implicitly imported into every Pollen markup file. And if you don’t know what it does, you might end up surprised by some of the behavior you get.
In the Pollen markup environment, however, this behavior is annoying. Because when you’re writing X-expressions, you don’t necessarily want to define all your tags ahead of time.
So Pollen redefines #%top. For convenience, Pollen’s version of #%top assumes that an undefined tag should just refer to an X-expression beginning with that tag (and uses make-tag-function to provide this behavior):
Examples:
;Again, let's call em without defining it, but using pollen/top
The good news is that this behavior means you use any tag you want in your markup without defining it in advance. You can still attach a function to the tag later, which will automatically supersede #%top.
This isn’t a bug. It’s just a natural consequence of how Pollen’s #%top works. It can, however, make debugging difficult sometimes. Let’s suppose my markup depends on very-important-function, which I don’t import correctly.
So the undefined-function bug goes unreported. Again, that’s not a bug in Pollen — there’s just no way for it to tell the difference between an identifier that’s deliberately undefined and one that’s inadvertently undefined. If you want to guarantee that you’re invoking a defined identifier, use def/c.
syntax
(def/cid)
Invoke id if it’s a defined identifier, otherwise raise an error. This form reverses the behavior of #%top (in other words, it restores default Racket behavior).
Recall this example from before. In standard Racket, you get an undefined-identifier error.
You’ll probably never invoke this module directly. But it’s implicitly imported into every Pollen markup file. And if you don’t know what it does, you might end up surprised by some of the behavior you get.
In the Pollen markup environment, however, this behavior is annoying. Because when you’re writing X-expressions, you don’t necessarily want to define all your tags ahead of time.
So Pollen redefines #%top. For convenience, Pollen’s version of #%top assumes that an undefined tag should just refer to an X-expression beginning with that tag (and uses make-tag-function to provide this behavior):
Examples:
;Again, let's call em without defining it, but using pollen/top
The good news is that this behavior means you use any tag you want in your markup without defining it in advance. You can still attach a function to the tag later, which will automatically supersede #%top.
This isn’t a bug. It’s just a natural consequence of how Pollen’s #%top works. It can, however, make debugging difficult sometimes. Let’s suppose my markup depends on very-important-function, which I don’t import correctly.
So the undefined-function bug goes unreported. Again, that’s not a bug in Pollen — there’s just no way for it to tell the difference between an identifier that’s deliberately undefined and one that’s inadvertently undefined. If you want to guarantee that you’re invoking a defined identifier, use def/c.
syntax
(def/cid)
Invoke id if it’s a defined identifier, otherwise raise an error. This form reverses the behavior of #%top (in other words, it restores default Racket behavior).
Recall this example from before. In standard Racket, you get an undefined-identifier error.
\ No newline at end of file
diff --git a/doc/World.html b/doc/World.html
index c774c8c..a499db1 100644
--- a/doc/World.html
+++ b/doc/World.html
@@ -1,5 +1,5 @@
-10.9 World
A parameter that determines whether the world:project-require file is checked for changes on every pass through render. (Can be faster to turn this off if you don’t need it.) Initialized to #t.
A parameter that reports the path to the directory of support files for the project server. Initialized to #f, but set to a proper value when the server runs.
A parameter that determines whether the world:project-require file is checked for changes on every pass through render. (Can be faster to turn this off if you don’t need it.) Initialized to #t.
A parameter that reports the path to the directory of support files for the project server. Initialized to #f, but set to a proper value when the server runs.
A summary of the key components & concepts of the Pollen publishing system and how they fit together. If you’ve completed the Quick tour, this will lend some context to what you saw. The next tutorials will make more sense if you read this first.
4.1The book is a program
This is the core design principle of Pollen. Consistent with this principle, Pollen adopts the habits of software development in its functionality, workflow, and project management.
You are a programmer. Don’t panic. But let’s just admit it — if your book is a program, then you are, in part, programming it. You don’t have to know any programming to start using Pollen. But you’ll have to be willing to learn a few programming ideas. (Those who have programmed other template-based HTML generators may have to forget a few things.)
A Pollen project consists of source files + static files. A source file is a file that can be compiled to produce certain output. A static file is usable as it stands (e.g., an SVG file or webfont). Generally, the textual content of your book will live in source files, and other elements will be static files.
Source control is a good idea. Because Pollen projects are software projects, they can be easily managed with systems for source control and collaboration, like GitHub. If you’re a writer at heart, don’t fear these systems — the learning curve is repaid by revision & edit tracking that’s much easier than it is with Word or PDF files.
4.2One language, multiple dialects
Everything is Racket. The Pollen system is built entirely in the Racket programming language. Some of your source files will be in Racket. Others will be in one of the Pollen language dialects. But under the hood, everything becomes Racket code. So if you plan to do any serious work in Pollen, you’ll want to learn some basics about Racket too (for instance Quick: An Introduction to Racket with Pictures).
The Pollen language is based on Scribble. Scribble is a variant of the Racket language that flips the usual programming syntax: instead of code with embedded textual content, a Scribble source file is text with embedded code (an idea borrowed from TeX). The Pollen language is adapted from Scribble. So most things that are true about Scribble are also true about Pollen (see Scribble: The Racket Documentation Tool).
The Pollen language is divided into dialects. The Pollen dialects share a common syntax and structure. But they’re different in details that makes them better adapted to certain types of source files (for instance, one dialect of Pollen understands Markdown; the others don’t). Use whichever suits the task at hand.
4.3Development environment
The Pollen development environment has three main pieces: the DrRacket code editor, the project server, and the command-line tool.
Edit source files with DrRacket. DrRacket is Racket’s GUI code editor. Sure, you can also use a generic text editor. But DrRacket lets you immediately run your source and see if it works.
Preview & test web pages with the Pollen project server. Pollen has a built-in development web server called the project server. After you start the project server, you can preview your web pages within any web browser, allowing you to test them with maximum accuracy.
Write the docs. The project server can recognize and render Scribble files, so you can use it as a previewing tool while you’re writing your documentation.
Render & deploy from the command line. Your Pollen project ultimately gets rendered to a set of static files (usually HTML and related assets). This can be controlled from the command line, so you can integrate it into other scripts.
4.4A special data structure for HTML
Unlike other programming languages, Pollen (and Racket) internally represent HTML with something called an X-expression. An X-expression is simply a list that represents what in HTML is called an element, meaning a thing with an opening tag, a closing tag, and content in between. Like HTML elements, X-expressions can be nested. Unlike HTML elements, X-expressions have no closing tag, they use parentheses to denote the start and end, and text elements are put inside quotes.
For example, consider this HTML element:
<body><h1>Hello world</h1><p>Nice to <i>see</i> you.</p></body>
As a Racket X-expression, this would be written:
(body (h1 "Hello world") (p "Nice to " (i "see") " you."))
More will be said about X-expressions. But a couple advantages should be evident already. First, without the redundant angle brackets, the X-expression is more readable than the equivalent HTML. Second, an X-expression is preferable to representing HTML as a simple string, because it preserves the internal structure of the element.
4.5Pollen command syntax
As mentioned above, a Pollen source file is not code with text embedded in it, but rather text with code embedded. (See ◊ command overview for more.)
If you can write text, you can program in Pollen. Really. As you already found out in the Quick tour, this is a valid Pollen program:
+4 The big picture
A summary of the key components & concepts of the Pollen publishing system and how they fit together. If you’ve completed the Quick tour, this will lend some context to what you saw. The next tutorials will make more sense if you read this first.
4.1The book is a program
This is the core design principle of Pollen. Consistent with this principle, Pollen adopts the habits of software development in its functionality, workflow, and project management.
You are a programmer. Don’t panic. But let’s just admit it — if your book is a program, then you are, in part, programming it. You don’t have to know any programming to start using Pollen. But you’ll have to be willing to learn a few programming ideas. (Those who have programmed other template-based HTML generators may have to forget a few things.)
A Pollen project consists of source files + static files. A source file is a file that can be compiled to produce certain output. A static file is usable as it stands (e.g., an SVG file or webfont). Generally, the textual content of your book will live in source files, and other elements will be static files.
Source control is a good idea. Because Pollen projects are software projects, they can be easily managed with systems for source control and collaboration, like GitHub. If you’re a writer at heart, don’t fear these systems — the learning curve is repaid by revision & edit tracking that’s much easier than it is with Word or PDF files.
4.2One language, multiple dialects
Everything is Racket. The Pollen system is built entirely in the Racket programming language. Some of your source files will be in Racket. Others will be in one of the Pollen language dialects. But under the hood, everything becomes Racket code. So if you plan to do any serious work in Pollen, you’ll want to learn some basics about Racket too (for instance Quick: An Introduction to Racket with Pictures).
The Pollen language is based on Scribble. Scribble is a variant of the Racket language that flips the usual programming syntax: instead of code with embedded textual content, a Scribble source file is text with embedded code (an idea borrowed from TeX). The Pollen language is adapted from Scribble. So most things that are true about Scribble are also true about Pollen (see Scribble: The Racket Documentation Tool).
The Pollen language is divided into dialects. The Pollen dialects share a common syntax and structure. But they’re different in details that makes them better adapted to certain types of source files (for instance, one dialect of Pollen understands Markdown; the others don’t). Use whichever suits the task at hand.
4.3Development environment
The Pollen development environment has three main pieces: the DrRacket code editor, the project server, and the command-line tool.
Edit source files with DrRacket. DrRacket is Racket’s GUI code editor. Sure, you can also use a generic text editor. But DrRacket lets you immediately run your source and see if it works.
Preview & test web pages with the Pollen project server. Pollen has a built-in development web server called the project server. After you start the project server, you can preview your web pages within any web browser, allowing you to test them with maximum accuracy.
Write the docs. The project server can recognize and render Scribble files, so you can use it as a previewing tool while you’re writing your documentation.
Render & deploy from the command line. Your Pollen project ultimately gets rendered to a set of static files (usually HTML and related assets). This can be controlled from the command line, so you can integrate it into other scripts.
4.4A special data structure for HTML
Unlike other programming languages, Pollen (and Racket) internally represent HTML with something called an X-expression. An X-expression is simply a list that represents what in HTML is called an element, meaning a thing with an opening tag, a closing tag, and content in between. Like HTML elements, X-expressions can be nested. Unlike HTML elements, X-expressions have no closing tag, they use parentheses to denote the start and end, and text elements are put inside quotes.
For example, consider this HTML element:
<body><h1>Hello world</h1><p>Nice to <i>see</i> you.</p></body>
As a Racket X-expression, this would be written:
(body (h1 "Hello world") (p "Nice to " (i "see") " you."))
More will be said about X-expressions. But a couple advantages should be evident already. First, without the redundant angle brackets, the X-expression is more readable than the equivalent HTML. Second, an X-expression is preferable to representing HTML as a simple string, because it preserves the internal structure of the element.
4.5Pollen command syntax
As mentioned above, a Pollen source file is not code with text embedded in it, but rather text with code embedded. (See ◊ command overview for more.)
If you can write text, you can program in Pollen. Really. As you already found out in the Quick tour, this is a valid Pollen program:
Commands start with ◊. A simple rule: if a piece of text starts with ◊, it’s treated as a command; otherwise it’s treated as ordinary text.
Write commands in text mode or Racket mode. Commands can use two equivalent notation systems: either Pollen’s text-mode command syntax, or standard Racket syntax.
Everything in Racket is in Pollen too. This isn’t some dimwit “template language.” Racket is a fully provisioned programming language, and every Racket function is available in Pollen.
4.6The preprocessor
The preprocessor is the simplest processing mode in Pollen.
Text output. The preprocessor scans the source for any Pollen commands, resolves them, and outputs the whole file as text.
Work with any text file. I hope this blows your mind a teeny bit. You can use the preprocessor with HTML, CSS, Markdown, JavaScript, XML, SVG, or any other text-based file (including source files of other programming languages).
Start quickly. Because it works with any text file, the preprocessor is an easy way to try out Pollen, because you can mix it into your workflow on an existing project, or even just one file.
4.7Templated source files
If you want to apply a particular page format to multiple sources of content — as you would in a book — you can use Pollen templates.
Templates can be any format. Usually Pollen templates will be HTML. But they don’t have to be.
Markdown support. Pollen has a built-in Markdown parser, so you can import Markdown sources into a Pollen publication.
Custom markup. Pollen’s markup mode allows you the freedom to define your own markup tags and attach behavior to them.
Mix source types. Every text source is converted to an X-expression before going into a template. So it’s fine to have multiple kinds of text source in one project.
4.8Pagetrees
Similar to a table of contents, a pagetree is a special Pollen source file that gets turned into a hierarchical list of pages.
Navigation. Pagetrees are used to provide navigation links within HTML templates (like previous, next, up, top).
Organization. Multiple pagetrees can be used to divide your project into subsets of pages that should be treated separately.
\ No newline at end of file
diff --git a/doc/file-types.html b/doc/file-types.html
index 7314dec..cc51fa1 100644
--- a/doc/file-types.html
+++ b/doc/file-types.html
@@ -1,2 +1,2 @@
-10.3 File
A utility module that provides functions for working with Pollen source and output files. The tests rely on file extensions specified in pollen/world.
Pollen handles six kinds of source files:
Preprocessor, with file extension .pp.
Markup, with file extension .pm.
Template, with file extension .pt.
Null, with file extension .p.
Scribble, with file extension .scrbl.
For each kind of Pollen source file, the corresponding output file is generated by removing the extension from the name of the source file. So the preprocessor source file default.css.pp would become default.css. Scribble files work differently — the corresponding output file is the source file but with an html extension rather than scrbl. So pollen.scrbl would become pollen.html.
Test whether v is a path representing a source file of the specified type, or is the output path for an existing source file of the specified type. In other words, has/is-preproc-source? is equivalent to (or(preproc-source?v)(has-preproc-source?v)).
Convert an output path p into the source path of the specified type that would produce this output path. This function simply generates a path for a file — it does not ask whether the file exists.
Convert a source path p into its corresponding output path. This function simply generates a path for a file — it does not ask whether the file exists.
A utility module that provides functions for working with Pollen source and output files. The tests rely on file extensions specified in pollen/world.
Pollen handles six kinds of source files:
Preprocessor, with file extension .pp.
Markup, with file extension .pm.
Template, with file extension .pt.
Null, with file extension .p.
Scribble, with file extension .scrbl.
For each kind of Pollen source file, the corresponding output file is generated by removing the extension from the name of the source file. So the preprocessor source file default.css.pp would become default.css. Scribble files work differently — the corresponding output file is the source file but with an html extension rather than scrbl. So pollen.scrbl would become pollen.html.
Test whether v is a path representing a source file of the specified type, or is the output path for an existing source file of the specified type. In other words, has/is-preproc-source? is equivalent to (or(preproc-source?v)(has-preproc-source?v)).
Convert an output path p into the source path of the specified type that would produce this output path. This function simply generates a path for a file — it does not ask whether the file exists.
Convert a source path p into its corresponding output path. This function simply generates a path for a file — it does not ask whether the file exists.
\ No newline at end of file
diff --git a/doc/first-tutorial.html b/doc/first-tutorial.html
index cf75070..eaa8166 100644
--- a/doc/first-tutorial.html
+++ b/doc/first-tutorial.html
@@ -1,5 +1,5 @@
-5 First tutorial
In this tutorial, you’ll use Pollen to make a single HTML page with a poem. You’ll learn about:
The relationship of Racket & Pollen
DrRacket
The project server
The preprocessor
If you want the shortest possible introduction to Pollen, try the Quick tour.
5.1Prerequisites
I’m going to assume that you’ve already installed Racket and Pollen. If not, do that now.
I’m also going to assume you know the basics of using a command line to run programs and navigate the file system using commands like cd and ls. On Mac OS X, your command-line program is called Terminal; on Windows it’s the Windows Command Processor.
5.2The relationship of Racket & Pollen
As I mentioned in the The big picture, Pollen is built using Racket, and everything in Pollen ultimately becomes Racket code. If you’re comfortable with that idea, you may move along.
But if not, or if you’re just a curious character:
One of the key features of Racket as a programming language is that it provides tools to create other programming languages. These languages might look & behave like Racket. Or they might not. These languages might serve a general purpose, but more often they’re specialized for a particular purpose, in which case they’re known as domain-specific languages, or DSLs.
Racket exploits the fact that under the hood, all programming languages are basically doing the same thing. (CS jocks know this more formally as a side effect of Turing completeness.) Racket starts with the most general expression of a Turing-complete language — called the lambda calculus — and lets users build on that. In most programming languages, you can build functions, classes, and modules. But in Racket, you can alter anything about the language.
If you find this a strange idea, you’re not alone. Most programmers — and until recently, me too — have never made or used DSLs. If you have a programming problem to solve, you start with a general-purpose language like Python or Java or Ruby, and go from there. Nothing wrong with that.
But programming languages contain their own design choices and compromises. Sometimes the problem at hand is best solved by manipulating the language at a deeper level. When you make a DSL, you’re still programming in the underlying language, but doing so at a point of higher leverage.
Pollen is a DSL implemented in Racket. It is a close cousin of Scribble: The Racket Documentation Tool, another Racket DSL, which was designed for writing Racket documentation. The key feature of Scribble, and thus also of Pollen, is that it’s text-based. Meaning, whereas most languages have source files made of code with text embedded within, Pollen’s source files are text with code embedded within.
Moreover, Pollen is meant to be a small step away from Racket — you can think of it as a more convenient notation system for Racket code, similar to how Markdown is a more convenient notation for HTML. But unlike Markdown, which only lets you access a subset of HTML, anything that can be done in Racket can also be done in Pollen.
As you work more with Pollen, you’ll pick up more about how Pollen corresponds to Racket (see ◊ command overview) and easily be able to convert commands from one system to the other. In later tutorials, you’ll see how larger Pollen projects are made out of both Pollen and Racket source files.
But in smaller projects, like this one, you can just use Pollen.
5.3Starting a new file in DrRacket
DrRacket is the IDE for the Racket programming language, and other languages made with Racket (like Pollen). IDE stands for “Integrated Development Environment,” which is a fancy phrase for “a nice place to edit and run your code.” DrRacket is installed as part of the core Racket distribution.
If you’ve worked with languages like Perl, Python, or Ruby, you may be more familiar with using a general-purpose text editor to edit your code, and then running your program at the command line. You can do that with Racket too. But DrRacket is a considerately designed tool. I recommend it. For these tutorials, I’ll assume you’re using DrRacket. If you insist on using the command line, I trust you to figure out what you need to do to keep up.
Launch DrRacket. Start a new file. The code in the file will look like this:
Within the main window, you should also see an interactions window, which shows the output of the current file, and starts out looking something like this (details, like the version number, will vary):
Welcome to DrRacket, version 6.0.1.6--2013-11-26(-/f) [3m].
Language: racket; memory limit: 1000 MB.
>
If you don’t see the interactions window, select View|Show Interactions from the menu.
5.3.1Setting the #lang line
The first line of every Racket source file, and every Pollen source file, is called the #lang line. The #lang line identifies the language used to interpret the rest of the file.
When you start a new Pollen source file in DrRacket, you’ll need to change the #lang line to the Pollen language. The simplest way is to change the first line to this:
In this tutorial, you’ll use Pollen to make a single HTML page with a poem. You’ll learn about:
The relationship of Racket & Pollen
DrRacket
The project server
The preprocessor
If you want the shortest possible introduction to Pollen, try the Quick tour.
5.1Prerequisites
I’m going to assume that you’ve already installed Racket and Pollen. If not, do that now.
I’m also going to assume you know the basics of using a command line to run programs and navigate the file system using commands like cd and ls. On Mac OS X, your command-line program is called Terminal; on Windows it’s the Windows Command Processor.
5.2The relationship of Racket & Pollen
As I mentioned in the The big picture, Pollen is built using Racket, and everything in Pollen ultimately becomes Racket code. If you’re comfortable with that idea, you may move along.
But if not, or if you’re just a curious character:
One of the key features of Racket as a programming language is that it provides tools to create other programming languages. These languages might look & behave like Racket. Or they might not. These languages might serve a general purpose, but more often they’re specialized for a particular purpose, in which case they’re known as domain-specific languages, or DSLs.
Racket exploits the fact that under the hood, all programming languages are basically doing the same thing. (CS jocks know this more formally as a side effect of Turing completeness.) Racket starts with the most general expression of a Turing-complete language — called the lambda calculus — and lets users build on that. In most programming languages, you can build functions, classes, and modules. But in Racket, you can alter anything about the language.
If you find this a strange idea, you’re not alone. Most programmers — and until recently, me too — have never made or used DSLs. If you have a programming problem to solve, you start with a general-purpose language like Python or Java or Ruby, and go from there. Nothing wrong with that.
But programming languages contain their own design choices and compromises. Sometimes the problem at hand is best solved by manipulating the language at a deeper level. When you make a DSL, you’re still programming in the underlying language, but doing so at a point of higher leverage.
Pollen is a DSL implemented in Racket. It is a close cousin of Scribble: The Racket Documentation Tool, another Racket DSL, which was designed for writing Racket documentation. The key feature of Scribble, and thus also of Pollen, is that it’s text-based. Meaning, whereas most languages have source files made of code with text embedded within, Pollen’s source files are text with code embedded within.
Moreover, Pollen is meant to be a small step away from Racket — you can think of it as a more convenient notation system for Racket code, similar to how Markdown is a more convenient notation for HTML. But unlike Markdown, which only lets you access a subset of HTML, anything that can be done in Racket can also be done in Pollen.
As you work more with Pollen, you’ll pick up more about how Pollen corresponds to Racket (see ◊ command overview) and easily be able to convert commands from one system to the other. In later tutorials, you’ll see how larger Pollen projects are made out of both Pollen and Racket source files.
But in smaller projects, like this one, you can just use Pollen.
5.3Starting a new file in DrRacket
DrRacket is the IDE for the Racket programming language, and other languages made with Racket (like Pollen). IDE stands for “Integrated Development Environment,” which is a fancy phrase for “a nice place to edit and run your code.” DrRacket is installed as part of the core Racket distribution.
If you’ve worked with languages like Perl, Python, or Ruby, you may be more familiar with using a general-purpose text editor to edit your code, and then running your program at the command line. You can do that with Racket too. But DrRacket is a considerately designed tool. I recommend it. For these tutorials, I’ll assume you’re using DrRacket. If you insist on using the command line, I trust you to figure out what you need to do to keep up.
Launch DrRacket. Start a new file. The code in the file will look like this:
Within the main window, you should also see an interactions window, which shows the output of the current file, and starts out looking something like this (details, like the version number, will vary):
Welcome to DrRacket, version 6.0.1.6--2013-11-26(-/f) [3m].
Language: racket; memory limit: 1000 MB.
>
If you don’t see the interactions window, select View|Show Interactions from the menu.
5.3.1Setting the #lang line
The first line of every Racket source file, and every Pollen source file, is called the #lang line. The #lang line identifies the language used to interpret the rest of the file.
When you start a new Pollen source file in DrRacket, you’ll need to change the #lang line to the Pollen language. The simplest way is to change the first line to this:
Now run your file by clicking the Run button in the upper-right corner, or select Racket|Run from the menu. You’ll get something like:
Welcome to DrRacket, version 6.0.1.6--2013-11-26(-/f) [3m].
Language: pollen; memory limit: 1000 MB.
>
Notice that the language is now reported as pollen. If you like, change the #lang line to this:
#lang pollenxyz
Then click Run again. DrRacket will print an error:
Module Language: invalid module text
standard-module-name-resolver: collection not found ...
Why? Because there’s no language called pollenxyz. Switch it back to pollen and let’s move on.
5.3.2Putting in the text of the poem
Here’s a short, bad poem I wrote about CSS.
The margin is 42em.
The border is red.
The padding is 15em.
The border is too.
Paste the text of this poem into your DrRacket editing window, below the #lang line, so it looks like this:
#lang pollen
The margin is 42em.
The border is red.
The padding is 15em.
The border is too.
Run the file again. In the interactions window, you’ll see:
The margin is 8em.
The border is blue.
The padding is 2em.
diff --git a/doc/index.html b/doc/index.html
index 049a712..bc1ad05 100644
--- a/doc/index.html
+++ b/doc/index.html
@@ -1,3 +1,3 @@
-Pollen: the book is a program
Pollen is a publishing system that helps authors create beautiful and functional web-based books. Pollen includes tools for writing, designing, programming, testing, and publishing.
I used Pollen to create my book Butterick’s Practical Typography. Sure, go take a look. Is it better than the last digital book you encountered? Yes it is. Would you like your book to look like that? If so, keep reading.
At the core of Pollen is an argument:
+Pollen: the book is a program
Pollen is a publishing system that helps authors create beautiful and functional web-based books. Pollen includes tools for writing, designing, programming, testing, and publishing.
I used Pollen to create my book Butterick’s Practical Typography. Sure, go take a look. Is it better than the last digital book you encountered? Yes it is. Would you like your book to look like that? If so, keep reading.
At the core of Pollen is an argument:
First, that digital books should be the best books we’ve ever had. So far, they’re not even close.
Second, that because digital books are software, an author shouldn’t think of a book as merely data. The book is a program.
Third, that the way we make digital books better than their predecessors is by exploiting this programmability.
That’s what Pollen is for.
Not that you need to be a programmer to use Pollen. On the contrary, the Pollen language is markup-based, so you can write & edit text naturally. But when you want to automate repetitive tasks, add cross-references, or pull in data from other sources, you can access a full programming language from within the text.
That language is Racket. I chose Racket because while the idea for Pollen had been with me for several years, it simply wasn’t possible to build it with other languages. So if it’s unfamiliar to you, don’t panic. It was unfamiliar to me. Once you see what you can do with Pollen & Racket, you may be persuaded. I was.
Or, if you can find a better digital-publishing tool, use that. But I’m never going back to the way I used to work.
You don’t have to use Supertramp song titles. Any text will do. When you click Run again, you’ll see whatever you typed:
Goodbye Stranger Breakfast in America Take the Long Way Home
We won’t do it a third time. You get the point — any plain text is valid within a Pollen source file, and gets printed as is. You never have to perform the incantations of typical programming languages:
print "Hello world"
document.write('Hello world');
printf("Hello world");
In Pollen, what you write is what you get.
2.3Naming, saving, and rendering a source file
Save this file with the name hello.txt.pp in any convenient directory. The desktop is fine.
Open a terminal window and issue two commands:
> cd [directory containing your file]
> raco pollen render hello.txt.pp
After a moment, a new file will appear called hello.txt. Let’s see what’s in it:
> cat hello.txt
Goodbye Stranger
Breakfast in America
Take the Long Way Home
You’ve just learned three things:
Pollen commands in the terminal begin with raco pollen, followed by a specific command (in this case render) and sometimes an argument (in this case hello.txt.pp).
The render command takes the ouput from your source file — meaning, the result you previewed in DrRacket in the previous step — and saves it to another file.
The name of the output file is the same as the source file, minus the Pollen source extension. So hello.txt.pp becomes hello.txt.
Try editing the text in the hello.txt.pp source file and running raco pollen render hello.txt.pp again. The old hello.txt will be replaced with a new one showing your changes. And so you’ve learned a fourth thing:
Pollen works by rendering output files from source files. Output files can be overwritten. Therefore, you should only make edits to your source files.
2.4The project server
You’ve just learned two ways to see the output of a Pollen source file — first, you ran it in DrRacket. Then, you rendered it to an output file.
Now here’s a third: the Pollen project server. Here’s how you start it. Return to your terminal window and issue two commands:
> cd [directory containing your hello.txt.pp file]
> raco pollen start
After a moment, you’ll see the startup message:
Welcome to Pollen 0.001 (Racket 6.0.0.5)
Project root is /path/to/your/directory
Project server is http://localhost:8080 (Ctrl-C to exit)
Project dashboard is http://localhost:8080/index.ptree
Ready to rock
Open a web browser and point it at http://localhost:8080/index.ptree. The top of the window will say Project root. Below that will be a listing of the files in the directory.
Among them will be hello.txt, with a greyed-out .pp extension. Click on it, and you’ll be taken to http://localhost:8080/hello.txt, where you’ll see:
Goodbye Stranger
Breakfast in America
Take the Long Way Home
That’s the boring part. Here’s the good part. Leave the project server running. Open your source file again in DrRacket and edit it as follows:
Notice what happened — the Pollen project server dynamically regenerated the output file (hello.txt) from the source file (hello.txt.pp) after you edited the source. If you like, try making some more changes to hello.txt.pp, and reloading the browser to see the updates in hello.txt.
2.5Intermission
That covers input & output. Now let’s circle back and look at what else you can do with Pollen (beyond the epic achievement of displaying plain text in a web browser).
For the rest of this tutorial, I recommend keeping two windows on screen: a web-browser window pointed at your project server (the main URL is http://localhost:8080/index.ptree) and the DrRacket editing window.
2.6Pollen as a preprocessor
A preprocessor is a tool for making systematic, automated changes to a source file before the main processing happens. A preprocessor can also be used to add programming logic to files that otherwise don’t support it.
For instance, HTML. In DrRacket, create a new file called margin.html.pp in your project directory:
The “.pp” file extension — which you saw before, with hello.txt.pp — stands for “Pollen preprocessor.” You can use the Pollen preprocessor with any text-based file by inserting #lang pollen as the first line, and adding the .pp file extension.
But for now, go to your project dashboard and click on margin.html. You should see a black box containing the text “5em is the inset.”
Let’s suppose you want to change the inset to 30%. Without a preprocessor, you’d have to search & replace each value. But with a preprocessor, you can move the inset value into a variable, and update it from that one location. So first, introduce a variable called my-inset by using the define command:
The ◊ character is called a lozenge. In Pollen, the lozenge is a special character that marks anything Pollen should interpret as a command (rather than plain text).
You don’t have to use Supertramp song titles. Any text will do. When you click Run again, you’ll see whatever you typed:
Goodbye Stranger Breakfast in America Take the Long Way Home
We won’t do it a third time. You get the point — any plain text is valid within a Pollen source file, and gets printed as is. You never have to perform the incantations of typical programming languages:
print "Hello world"
document.write('Hello world');
printf("Hello world");
In Pollen, what you write is what you get.
2.3Naming, saving, and rendering a source file
Save this file with the name hello.txt.pp in any convenient directory. The desktop is fine.
Open a terminal window and issue two commands:
> cd [directory containing your file]
> raco pollen render hello.txt.pp
After a moment, a new file will appear called hello.txt. Let’s see what’s in it:
> cat hello.txt
Goodbye Stranger
Breakfast in America
Take the Long Way Home
You’ve just learned three things:
Pollen commands in the terminal begin with raco pollen, followed by a specific command (in this case render) and sometimes an argument (in this case hello.txt.pp).
The render command takes the ouput from your source file — meaning, the result you previewed in DrRacket in the previous step — and saves it to another file.
The name of the output file is the same as the source file, minus the Pollen source extension. So hello.txt.pp becomes hello.txt.
Try editing the text in the hello.txt.pp source file and running raco pollen render hello.txt.pp again. The old hello.txt will be replaced with a new one showing your changes. And so you’ve learned a fourth thing:
Pollen works by rendering output files from source files. Output files can be overwritten. Therefore, you should only make edits to your source files.
2.4The project server
You’ve just learned two ways to see the output of a Pollen source file — first, you ran it in DrRacket. Then, you rendered it to an output file.
Now here’s a third: the Pollen project server. Here’s how you start it. Return to your terminal window and issue two commands:
> cd [directory containing your hello.txt.pp file]
> raco pollen start
After a moment, you’ll see the startup message:
Welcome to Pollen 0.001 (Racket 6.0.0.5)
Project root is /path/to/your/directory
Project server is http://localhost:8080 (Ctrl-C to exit)
Project dashboard is http://localhost:8080/index.ptree
Ready to rock
Open a web browser and point it at http://localhost:8080/index.ptree. The top of the window will say Project root. Below that will be a listing of the files in the directory.
Among them will be hello.txt, with a greyed-out .pp extension. Click on it, and you’ll be taken to http://localhost:8080/hello.txt, where you’ll see:
Goodbye Stranger
Breakfast in America
Take the Long Way Home
That’s the boring part. Here’s the good part. Leave the project server running. Open your source file again in DrRacket and edit it as follows:
Notice what happened — the Pollen project server dynamically regenerated the output file (hello.txt) from the source file (hello.txt.pp) after you edited the source. If you like, try making some more changes to hello.txt.pp, and reloading the browser to see the updates in hello.txt.
2.5Intermission
That covers input & output. Now let’s circle back and look at what else you can do with Pollen (beyond the epic achievement of displaying plain text in a web browser).
For the rest of this tutorial, I recommend keeping two windows on screen: a web-browser window pointed at your project server (the main URL is http://localhost:8080/index.ptree) and the DrRacket editing window.
2.6Pollen as a preprocessor
A preprocessor is a tool for making systematic, automated changes to a source file before the main processing happens. A preprocessor can also be used to add programming logic to files that otherwise don’t support it.
For instance, HTML. In DrRacket, create a new file called margin.html.pp in your project directory:
The “.pp” file extension — which you saw before, with hello.txt.pp — stands for “Pollen preprocessor.” You can use the Pollen preprocessor with any text-based file by inserting #lang pollen as the first line, and adding the .pp file extension.
But for now, go to your project dashboard and click on margin.html. You should see a black box containing the text “5em is the inset.”
Let’s suppose you want to change the inset to 30%. Without a preprocessor, you’d have to search & replace each value. But with a preprocessor, you can move the inset value into a variable, and update it from that one location. So first, introduce a variable called my-inset by using the define command:
The ◊ character is called a lozenge. In Pollen, the lozenge is a special character that marks anything Pollen should interpret as a command (rather than plain text).
How to type a lozenge:
Mac: option + shift + V
Windows: holding down alt, type 9674 on the num pad
Ubuntu: ctrl + shift + U, then 25CA
The whole command ◊define[my-inset]{30%} means “create a variable called my-inset and give it the value 30%.”
Then put the variable into the HTML like so, this time using the ◊ character with the variable name in the two places the value appears:
Now reload margin.html. You’ll see that the size of the margin has changed (because of the change to the style attribute) and so has the text of the HTML. If you like, try editing my-inset with different values and reloading the page. You can also try using define to create another variable (for instance, to change the color of the box border).
Still, this is the tiniest tip of the iceberg. The Pollen preprocessor gives you access to everything in the Racket programming language — including math functions, text manipulation, and so on.
2.7Markdown mode
When used as a preprocessor, Pollen’s rule is that what you write is what you get. But if you’re targeting HTML, who wants to type out all those <tedious>tags</tedious>? You can make Pollen do the heavy lifting by using it as a source decoder.
For instance, Markdown mode. Markdown is a simplified notation system for HTML. You can use Pollen’s Markdown decoder by inserting #lang pollen as the first line, and adding the .pmd file extension.
Try it. In DrRacket, create a file with the following lines and save it as downtown.html.pmd:
As usual, you’re welcome to edit downtown.html.pmd and then refresh the web browser to see the changes.
In Markdown mode, you can still embed Pollen commands within the source as you did in preprocessor mode. Just keep in mind that your commands need to produce valid Markdown (as opposed to raw HTML). For instance, use define to create a variable called metal, and insert it into the Markdown:
Pollen is handling two tasks here: interpreting the commands in the source, and then converting the Markdown to HTML. But what if you wanted to use Pollen as a preprocessor that outputs a Markdown file? No problem — just change the source name from downtown.html.pmd to downtown.md.pp. Changing the extension from .pmd to .pp switches Pollen from Markdown mode back to preprocessor mode. And changing the base name from downtown.html to downtown.md updates the name of the output file.
2.8Markup mode
If all you need to do is produce basic HTML, Markdown is great. But if you need to do semantic markup or other kinds of custom markup, it’s not flexible enough.
In that case, you can use Pollen markup mode. To use Pollen markup, insert #lang pollen as the first line of your source file, and add a .pm file extension.
Compared to Markdown mode, Pollen markup mode is wide open. Markdown mode gives you a limited set of formatting tools (i.e., the ones supported by Markdown). But in markup mode, you can use any tags you want. Markdown mode decodes the source in a fixed way (i.e., with the Markdown decoder). But markup mode lets you build any decoder you want.
Let’s convert our Markdown example into Pollen markup. Marking up content is simple: insert the lozenge character (◊) followed by the name of the tag (◊tag), followed by the content of the tag in curly braces (◊tag{content}). In DrRacket, create a new file called uptown.html.pm as follows:
Pollen markup You wanted it — you got it. https://google.com/search?q=racketsearch for Racket
That’s not right. What happened?
We marked up the source using a combination of standard HTML tags (strong, em) and nonstandard ones (headline, items, item, link). This is valid Pollen markup. (In fact, if you look at the generated source, you’ll see that they didn’t disappear.) But since we’re targeting HTML, we need to convert our custom tags into valid HTML tags.
For that, we’ll make a special file called project-require.rkt. This is a file in the standard Racket language that provides helper functions to decode the source. The definitions won’t make sense yet. But this is the quick tour, so all you need to do is copy, paste, and save:
Markup mode takes a little more effort to set up. But it also allows you more flexibility. If you want to do semantic markup, or convert your source into multiple output formats, or handle complex page layouts — it’s the way to go.
2.9Templates
The HTML pages we just made looked pretty dull. For the last stop on the quick tour, let’s fix that.
Pollen source files that are written in Markdown or markup mode (i.e., .pmd or .pm files) are rendered with a template. A template is not a standalone Pollen source file. It’s a file of the output type — e.g., CSS, HTML, XML — where you put the stuff that needs to be consistent between output files. The template also contains template variables that mark where values from the Pollen source file should be inserted.
When it needs a template, Pollen first looks for a file in the project directory named template.[output extension of source]. For uptown.html.pm, the output extension will be .html, thus Pollen will look for template.html.
So let’s create template.html. Make a new file that with the following lines and save it to the same directory as uptown.html.pm:
This is a simple HTML file that should look familiar, except for the two template variables. The first, here, contains the name of the current source file. As before, the lozenge character marks it as a Pollen command rather than text, so you write it as ◊here. The other command, ◊->html{◊doc}, takes the content from the source file, which is contained in a variable called doc, and converts it to HTML with a Pollen function called ->html.
Go back to your web browser and reload uptown.html. (Or downtown.html — both will work.) The page will be rendered with the new template.html. As before, you can edit the template or the source and the project server will dynamically update the output file.
2.10PS for Scribble users
Pollen can also be used as a dynamic preview server for Scribble files. From your terminal, do the following:
> cd [directory containing your Scribble files]
> raco pollen start
On the project dashboard, you’ll see your [filename].scrbl files listed as [filename].html. This may not represent the ultimate structure of your Scribble project — you may end up combining multiple Scribble source files into one HTML file, or making multiple HTML files from one Scribble source — but it’s handy for checking your work as you go.
2.11The end of the beginning
Now you’ve seen the key features of Pollen. What do you think?
“So it’s like WordPress, but harder to use?” I was a happy WordPress user for several years. If you need a blog, it’s great. But the farther you get from blogs, the more it becomes like teaching an elephant to pirouette. And for those who like to solve problems with programming, PHP is, um, limited.
“What about pairing a Python template system and Python web server?” Good idea. I even tried it. But Python template systems don’t offer you Python — they offer you pidgin dialects that ain’t very Pythonic. Also, Python’s handing of markup-based data structures is cumbersome.
“Haven’t you heard of Jekyll?” Yes. If everything you need to write is expressible in Markdown, it’s great. If you need more than that, you’re stuck.
“Sounds a lot like LaTeX. Why not use that?” Also a good idea. LaTeX gets a lot of things right. But it wasn’t designed for web publishing.
“Eh, there are plenty of adequate options. Why should I learn a system written in Racket, which I’ve never used?” A salient objection. It’s also the question I asked myself before I committed to Racket. But publishing systems that are author- or designer-friendly tend to be programmer-hostile, and vice versa. Racket is the only language I found that could meet my requirements.
But don’t take my word for it. The rest of this documentation will show you the cool, useful, and sophisticated things you can do with Pollen. If there’s another tool that suits you better, great. Keep in mind that I didn’t make Pollen because I’m a programmer. I’m a writer who wants to make electronic books that are better than the ones we have now. And for that, I needed a better tool.
\ No newline at end of file
diff --git a/doc/raco-pollen.html b/doc/raco-pollen.html
index 84185ce..0e25e64 100644
--- a/doc/raco-pollen.html
+++ b/doc/raco-pollen.html
@@ -1,2 +1,2 @@
-7 Using raco pollen
Racket provides centralized command-line options through raco (short for racketcommand, see raco: Racket Command-Line Tools).
Once you install Pollen, you can access the following Pollen-specific commands through raco using the subcommand raco pollen.
7.1Making sure racopollen works
Open a terminal window and type:
> raco pollen test
If racopollen is installed correctly, you’ll see:
raco pollen is installed correctly
But if you get:
raco: Unrecognized command: pollen
You’ll need to fix the problem before proceeding, most likely by reinstalling Pollen (see Installation).
Pro tip: I have an alias in my .bash_profile like so: alias polcom='raco pollen'
7.2racopollen
Same as racopollenhelp.
7.3racopollenhelp
Displays a list of available commands.
7.4racopollenstart
Starts the project server from the current directory using the default port, which is the value of the parameter world:current-server-port (by default, port 8080).
This command can be invoked with two optional arguments.
racopollenstartpath will start the project server in path rather than the current directory.
> raco pollen start ~/path/to/project/
racopollenstartpathport will start the project server in path using port rather than world:current-server-port. This is useful if you want to have multiple project servers running simultaneously.
If you want to start in the current directory but with a different port, use . as the path:
> raco pollen start . 8088
7.5racopollenrender
Renders all preprocessor source files and then all pagetree files found in the current directory.
This command can be invoked with extra arguments.
racopollenrenderdirectory will render the preprocessor source files and pagetree files in the specified directory.
Alternatively, the command can take a variable number of path arguments. racopollenrenderpath... will render only the paths specified in path.... Consistent with the usual command-line idiom, this can be a single path, a list of paths, or a pattern:
Racket provides centralized command-line options through raco (short for racketcommand, see raco: Racket Command-Line Tools).
Once you install Pollen, you can access the following Pollen-specific commands through raco using the subcommand raco pollen.
7.1Making sure racopollen works
Open a terminal window and type:
> raco pollen test
If racopollen is installed correctly, you’ll see:
raco pollen is installed correctly
But if you get:
raco: Unrecognized command: pollen
You’ll need to fix the problem before proceeding, most likely by reinstalling Pollen (see Installation).
Pro tip: I have an alias in my .bash_profile like so: alias polcom='raco pollen'
7.2racopollen
Same as racopollenhelp.
7.3racopollenhelp
Displays a list of available commands.
7.4racopollenstart
Starts the project server from the current directory using the default port, which is the value of the parameter world:current-server-port (by default, port 8080).
This command can be invoked with two optional arguments.
racopollenstartpath will start the project server in path rather than the current directory.
> raco pollen start ~/path/to/project/
racopollenstartpathport will start the project server in path using port rather than world:current-server-port. This is useful if you want to have multiple project servers running simultaneously.
If you want to start in the current directory but with a different port, use . as the path:
> raco pollen start . 8088
7.5racopollenrender
Renders all preprocessor source files and then all pagetree files found in the current directory.
This command can be invoked with extra arguments.
racopollenrenderdirectory will render the preprocessor source files and pagetree files in the specified directory.
Alternatively, the command can take a variable number of path arguments. racopollenrenderpath... will render only the paths specified in path.... Consistent with the usual command-line idiom, this can be a single path, a list of paths, or a pattern:
Pollen uses a special character — the lozenge, which looks like this: ◊ — to mark commands within a Pollen source file. So when you put a ◊ in your source, whatever comes next will be treated as a command. If you don’t, it will just be interpreted as plain text.
9.2The lozenge glyph (◊)
I chose the lozenge as the command marker because a) it appears in almost every font, b) it’s barely used in ordinary typesetting, c) it’s not used in any programming language that I know of, and d) its shape and color allow it to stand out easily in code without being distracting.
Pollen uses a special character — the lozenge, which looks like this: ◊ — to mark commands within a Pollen source file. So when you put a ◊ in your source, whatever comes next will be treated as a command. If you don’t, it will just be interpreted as plain text.
9.2The lozenge glyph (◊)
I chose the lozenge as the command marker because a) it appears in almost every font, b) it’s barely used in ordinary typesetting, c) it’s not used in any programming language that I know of, and d) its shape and color allow it to stand out easily in code without being distracting.
Here’s how you type it:
Mac: option + shift + V
Windows: holding down alt, type 9674 on the num pad
diff --git a/doc/second-tutorial.html b/doc/second-tutorial.html
index ace172d..e87dc7a 100644
--- a/doc/second-tutorial.html
+++ b/doc/second-tutorial.html
@@ -1,5 +1,5 @@
-
In this tutorial, you’ll use Pollen to publish a multiple-page article written in Markdown. You’ll learn about:
Using Markdown files with the preprocessor
X-expressions
Markdown authoring mode
Templates
Pagetrees
If you want the shortest possible introduction to Pollen, try the Quick tour.
6.1Prerequisites
I’ll assume you’ve completed the First tutorial and you understand how to create source files in DrRacket and view them in the project server. I will not be spelling out those tasks as I did before.
6.2Prelude: my principled objection to Markdown
I recognize that people like Markdown. I want people to like Pollen too, so that’s why Markdown support exists in Pollen. But just to be clear about my own views —
I’m mystified by the popularity of Markdown among writers. I can agree that it’s a clever and readable way of notating basic HTML. And sure, that makes it great for things like web comments, where speed and simplicity are primary virtues.
In longer-form writing, however, its shortcomings become evident. Like programming languages, the best writing tools maximize expressive possibilities, and minimize constraints. But Markdown is hugely constrained. First and worst, Markdown isn’t semantic. It only knows about formatting, and in that regard, isn’t that much of an improvement on tools like Microsoft Word. Second, even as a formatting-notation tool, it’s limited to a small subset of the already-small set of formatting tags permitted in HTML. Third, it can’t be extended by an author.
An animating principle of Pollen, as explained in the Backstory, is that after 20 years, we ought to move beyond thinking of HTML as a source format. Since Markdown is just well-disguised HTML, a vote for Markdown is really a vote to continue the status quo (albeit with fewer angle brackets). For me, that’s not good enough. I’m ready for the tools to expand to fit my ideas; I don’t want to keep cutting down my ideas to fit the tools.
All that said, if you genuinely prefer Markdown, I’m not looking to pry it from your fingers. Pollen has excellent Markdown support (due entirely to Greg Hendershott’s excellent Markdown parser for Racket). It makes Markdown more useful.
But let’s make a deal, Markdown fans. Having met you more than halfway, will you at least consider that Pollen markup might be a better option for you than Markdown? Because it can notate anything that’s in your brain, not just a subset of HTML? And if The book is a program, the source for that book should look more like your brain, and less like HTML?
That’s all I ask.
6.3Markdown in Pollen: two options
There are two ways to use Markdown within Pollen: you can either send Markdown files through the preprocessor, or use Markdown authoring mode.
The preprocessor approach is better if you want to end up with a set of Markdown files that can be passed along to a HTML converter (or other Markdown-to-______ converter) elsewhere.
The authoring-mode approach is better if you want to end up with something other than Markdown, e.g., finished HTML files.
6.3.1Using Markdown with the preprocessor
Because Markdown is a text-based format, you can use the Pollen preprocessor to add programmatic features to existing Markdown files. (See Working with the preprocessor in the First tutorial if you need a refresher.)
Suppose we have a Markdown file called brennan.md that we want to use with the preprocessor. Create this file in DrRacket, save it, and start the project server in that directory.
"brennan.md"
My name is _Brennan_, and I enjoy:
+ boring sauce
+ 24 fish nuggets
You’ll be able to see this file in the project server, but for now, it’s just a static file. Pollen isn’t doing anything to it.
Let’s change that. Consistent with the usual preprocessor practice, we add #lang pollen as the first line, and append the .pp file extension, so our new preprocessor-ready file looks like this:
"brennan.md.pp"
#lang pollen
My name is _Brennan_, and I enjoy:
+ boring sauce
+ 24 fish nuggets
Go back to the project server and you’ll see the new filename. When you click on it, Pollen will render a new markdown.md file, but it will look the same as the one you had before.
Now we’ll change some of the values using Pollen commands:
"brennan.md.pp"
#lang pollen
◊(define sauce-type "fancy")
◊(define nugget-type "chicken")
◊(define nugget-quantity (* 2 2 3))
My name is _Brennan_, and I enjoy:
+ ◊sauce-type sauce
+ ◊nugget-quantity ◊nugget-type nuggets
When you reload this file in the project server, brennan.md will be regenerated, and will now look like this:
My name is _Brennan_, and I enjoy:
+ fancy sauce
+ 12 chicken nuggets
Instead of running Markdown files through the preprocessor, you can also use Markdown authoring mode within Pollen. This is the better choice if you want to end up with rendered HTML files.
But first, let’s pause to clarify the general concept of an authoring mode.
6.3.2Authoring mode
Though the preprocessor is useful, it limits you to inserting chunks of text at various positions into an existing file.
Pollen’s authoring mode, by contrast, parses the whole source file into a special data structure called an X-expression. You can then process the whole X-expression any way you like, and output to any format you like — or multiple formats — using a template.
Compared to the preprocessor, authoring mode offers more abstraction and flexibility. Of course, it’s also a bit more effort to set up.
Pollen offers two variants of authoring mode: one that uses Markdown syntax (which we’ll cover later in this tutorial) and the other that uses a free-form markup syntax (which we’ll cover in the next tutorial). In both cases, the basic idea is the same: parse the source into an X-expression, and then output it using a template.
6.3.3X-expressions
Don’t skip this part! It’s not the same old shit. And it describes a concept that’s key to how Pollen works.
I avoid nerdy jargon whenever possible. But in this case, the thing is called an X-expression throughout the Racket documentation, for good reasons. So I use the term too. Better to acclimate you now.
An X-expression is a way of representing markup-based data in code. X-expressions are indigenous to Lisp-based languages like Pollen and Racket. They don’t exist in Python or JavaScript or Ruby.
Let’s start with the part you’re familiar with. By “markup-based data,” I mean things like HTML and XML and SVG. The idea is that you have text-based data surrounded by tags. Each tag can also have its own attributes that are made of keys and values. Tags can contain other tags, thus creating a tree-like structure. Right? You know what I mean:
<body><h1>Hello world</h1><p class="first">Nice to <i>see</i> you.</p></body>
An X-expression is just a simplified, generalized method of notation for these data structures — much like Markdown is a simplified method of notation for HTML. To see the relationship, we’ll convert one into the other.
First, we change the angle brackets to parentheses, and only use them on the outside of tags:
(body (h1 Hello world /h1) (p class="first" Nice to (i see /i) you. /p) /body)
Then we get rid of the closing tags, which are superfluous, since the closing parenthesis marks the end of the tag perfectly well:
(body (h1 Hello world) (p class="first" Nice to (i see) you.))
However, this creates ambiguity between the name of the tag and the content. So we’ll put the content within quote marks:
S-expressions use prefix notation, where each pair of parentheses contains a list. The first element in the list names a function, and the other elements are the arguments to that function. (This is a review of Racket basics (if you’re not familiar).) X-expressions are just a minor adaptation of S-expression notation to represent markup, hence the name (the X is short for XML-like).
For handling markup-based data, X-expressions have some useful advantages compared to other methods:
Readability. X-expressions retain all the semantics of markup-based data while dispensing with the infamous verbosity.
A hybrid between a tree and a string. Most programming languages represent markup-based data either as a string or as an XML tree. Neither is a good choice. The string captures none of the internal structure of the data. An XML tree captures the structure, but conceals the sequential nature of the data elements. The X-expression gets both.
An ideal match for an expression-based programming language. Aside from some notational details, X-expressions are just a subset of S-expressions generally, which are the building block of Racket. Processing X-expressions in Racket maximizes flexibility and minimizes yak-shaving.
Given the close kinship between XML-ish data structures and Lisp-ish programming languages, I have no explanation why, during the Internet era, they have not been paired more often.
In Pollen’s authoring modes, your source file is parsed into an X-expression, which can then be processed further before being injected into a template & converted to output. As a first example, we’ll look at Markdown authoring mode.
6.3.4Markdown authoring mode
Let’s start putting together our article. For simplicity, I’m going to use unrealistically short sample texts. But you can use whatever Markdown content you want.
We want to use Markdown authoring mode to make a file that will ultimately be HTML. So consistent with Pollen file-naming conventions (see Saving & naming your source file), we’ll start with our desired output filename, article.html, and then append the Markdown authoring suffix, .pmd. So in DrRacket, start a new file called article.html.pmd and put some Markdown in it:
"article.html.pmd"
#lang pollen
Deep Thought
============
I am **so** happy to be writing this.
Before you preview this file in the project server, click the Run button in DrRacket just to see what the file produces. You’ll see something like this:
’(root (h1 ((id "my-article")) "Deep Thought") (p () "I am "
+6 Second tutorial
In this tutorial, you’ll use Pollen to publish a multiple-page article written in Markdown. You’ll learn about:
Using Markdown files with the preprocessor
X-expressions
Markdown authoring mode
Templates
Pagetrees
If you want the shortest possible introduction to Pollen, try the Quick tour.
6.1Prerequisites
I’ll assume you’ve completed the First tutorial and you understand how to create source files in DrRacket and view them in the project server. I will not be spelling out those tasks as I did before.
6.2Prelude: my principled objection to Markdown
I recognize that people like Markdown. I want people to like Pollen too, so that’s why Markdown support exists in Pollen. But just to be clear about my own views —
I’m mystified by the popularity of Markdown among writers. I can agree that it’s a clever and readable way of notating basic HTML. And sure, that makes it great for things like web comments, where speed and simplicity are primary virtues.
In longer-form writing, however, its shortcomings become evident. Like programming languages, the best writing tools maximize expressive possibilities, and minimize constraints. But Markdown is hugely constrained. First and worst, Markdown isn’t semantic. It only knows about formatting, and in that regard, isn’t that much of an improvement on tools like Microsoft Word. Second, even as a formatting-notation tool, it’s limited to a small subset of the already-small set of formatting tags permitted in HTML. Third, it can’t be extended by an author.
An animating principle of Pollen, as explained in the Backstory, is that after 20 years, we ought to move beyond thinking of HTML as a source format. Since Markdown is just well-disguised HTML, a vote for Markdown is really a vote to continue the status quo (albeit with fewer angle brackets). For me, that’s not good enough. I’m ready for the tools to expand to fit my ideas; I don’t want to keep cutting down my ideas to fit the tools.
All that said, if you genuinely prefer Markdown, I’m not looking to pry it from your fingers. Pollen has excellent Markdown support (due entirely to Greg Hendershott’s excellent Markdown parser for Racket). It makes Markdown more useful.
But let’s make a deal, Markdown fans. Having met you more than halfway, will you at least consider that Pollen markup might be a better option for you than Markdown? Because it can notate anything that’s in your brain, not just a subset of HTML? And if The book is a program, the source for that book should look more like your brain, and less like HTML?
That’s all I ask.
6.3Markdown in Pollen: two options
There are two ways to use Markdown within Pollen: you can either send Markdown files through the preprocessor, or use Markdown authoring mode.
The preprocessor approach is better if you want to end up with a set of Markdown files that can be passed along to a HTML converter (or other Markdown-to-______ converter) elsewhere.
The authoring-mode approach is better if you want to end up with something other than Markdown, e.g., finished HTML files.
6.3.1Using Markdown with the preprocessor
Because Markdown is a text-based format, you can use the Pollen preprocessor to add programmatic features to existing Markdown files. (See Working with the preprocessor in the First tutorial if you need a refresher.)
Suppose we have a Markdown file called brennan.md that we want to use with the preprocessor. Create this file in DrRacket, save it, and start the project server in that directory.
"brennan.md"
My name is _Brennan_, and I enjoy:
+ boring sauce
+ 24 fish nuggets
You’ll be able to see this file in the project server, but for now, it’s just a static file. Pollen isn’t doing anything to it.
Let’s change that. Consistent with the usual preprocessor practice, we add #lang pollen as the first line, and append the .pp file extension, so our new preprocessor-ready file looks like this:
"brennan.md.pp"
#lang pollen
My name is _Brennan_, and I enjoy:
+ boring sauce
+ 24 fish nuggets
Go back to the project server and you’ll see the new filename. When you click on it, Pollen will render a new markdown.md file, but it will look the same as the one you had before.
Now we’ll change some of the values using Pollen commands:
"brennan.md.pp"
#lang pollen
◊(define sauce-type "fancy")
◊(define nugget-type "chicken")
◊(define nugget-quantity (* 2 2 3))
My name is _Brennan_, and I enjoy:
+ ◊sauce-type sauce
+ ◊nugget-quantity ◊nugget-type nuggets
When you reload this file in the project server, brennan.md will be regenerated, and will now look like this:
My name is _Brennan_, and I enjoy:
+ fancy sauce
+ 12 chicken nuggets
Instead of running Markdown files through the preprocessor, you can also use Markdown authoring mode within Pollen. This is the better choice if you want to end up with rendered HTML files.
But first, let’s pause to clarify the general concept of an authoring mode.
6.3.2Authoring mode
Though the preprocessor is useful, it limits you to inserting chunks of text at various positions into an existing file.
Pollen’s authoring mode, by contrast, parses the whole source file into a special data structure called an X-expression. You can then process the whole X-expression any way you like, and output to any format you like — or multiple formats — using a template.
Compared to the preprocessor, authoring mode offers more abstraction and flexibility. Of course, it’s also a bit more effort to set up.
Pollen offers two variants of authoring mode: one that uses Markdown syntax (which we’ll cover later in this tutorial) and the other that uses a free-form markup syntax (which we’ll cover in the next tutorial). In both cases, the basic idea is the same: parse the source into an X-expression, and then output it using a template.
6.3.3X-expressions
Don’t skip this part! It’s not the same old shit. And it describes a concept that’s key to how Pollen works.
I avoid nerdy jargon whenever possible. But in this case, the thing is called an X-expression throughout the Racket documentation, for good reasons. So I use the term too. Better to acclimate you now.
An X-expression is a way of representing markup-based data in code. X-expressions are indigenous to Lisp-based languages like Pollen and Racket. They don’t exist in Python or JavaScript or Ruby.
Let’s start with the part you’re familiar with. By “markup-based data,” I mean things like HTML and XML and SVG. The idea is that you have text-based data surrounded by tags. Each tag can also have its own attributes that are made of keys and values. Tags can contain other tags, thus creating a tree-like structure. Right? You know what I mean:
<body><h1>Hello world</h1><p class="first">Nice to <i>see</i> you.</p></body>
An X-expression is just a simplified, generalized method of notation for these data structures — much like Markdown is a simplified method of notation for HTML. To see the relationship, we’ll convert one into the other.
First, we change the angle brackets to parentheses, and only use them on the outside of tags:
(body (h1 Hello world /h1) (p class="first" Nice to (i see /i) you. /p) /body)
Then we get rid of the closing tags, which are superfluous, since the closing parenthesis marks the end of the tag perfectly well:
(body (h1 Hello world) (p class="first" Nice to (i see) you.))
However, this creates ambiguity between the name of the tag and the content. So we’ll put the content within quote marks:
S-expressions use prefix notation, where each pair of parentheses contains a list. The first element in the list names a function, and the other elements are the arguments to that function. (This is a review of Racket basics (if you’re not familiar).) X-expressions are just a minor adaptation of S-expression notation to represent markup, hence the name (the X is short for XML-like).
For handling markup-based data, X-expressions have some useful advantages compared to other methods:
Readability. X-expressions retain all the semantics of markup-based data while dispensing with the infamous verbosity.
A hybrid between a tree and a string. Most programming languages represent markup-based data either as a string or as an XML tree. Neither is a good choice. The string captures none of the internal structure of the data. An XML tree captures the structure, but conceals the sequential nature of the data elements. The X-expression gets both.
An ideal match for an expression-based programming language. Aside from some notational details, X-expressions are just a subset of S-expressions generally, which are the building block of Racket. Processing X-expressions in Racket maximizes flexibility and minimizes yak-shaving.
Given the close kinship between XML-ish data structures and Lisp-ish programming languages, I have no explanation why, during the Internet era, they have not been paired more often.
In Pollen’s authoring modes, your source file is parsed into an X-expression, which can then be processed further before being injected into a template & converted to output. As a first example, we’ll look at Markdown authoring mode.
6.3.4Markdown authoring mode
Let’s start putting together our article. For simplicity, I’m going to use unrealistically short sample texts. But you can use whatever Markdown content you want.
We want to use Markdown authoring mode to make a file that will ultimately be HTML. So consistent with Pollen file-naming conventions (see Saving & naming your source file), we’ll start with our desired output filename, article.html, and then append the Markdown authoring suffix, .pmd. So in DrRacket, start a new file called article.html.pmd and put some Markdown in it:
"article.html.pmd"
#lang pollen
Deep Thought
============
I am **so** happy to be writing this.
Before you preview this file in the project server, click the Run button in DrRacket just to see what the file produces. You’ll see something like this:
’(root (h1 ((id "my-article")) "Deep Thought") (p () "I am "
(strong () "so") " happy to be writing this."))
You should now be able to recognize this as an X-expression. In authoring mode, Pollen parses your Markdown into the corresponding HTML entities, but then provides the data as an X-expression rather than finished HTML.
The empty parentheses () after p and strong signal that the tag’s attributes are empty. When you write an X-expression without attributes, these parentheses are optional — (tag () "text") and (tag "text") are equivalent — but Pollen will always print X-expressions this way.
From what you learned in the last section, it should be evident that this X-expression corresponds to HTML that looks like this:
<root><h1 id="my-article">Deep Thought</h1><p>I am <strong>so</strong> happy to be writing this.</p></root>
“But what’s this root tag? That’s not HTML.” An X-expression must have a root tag, so in the spirit of obviousness, every X-expression produced by a source file in authoring mode will start with root. If you don’t need it, you can discard it. But it also creates a useful hook for further processing, as we’ll see later.
By the way, as review, let’s remind ourselves how this is different from preprocessor mode. Let’s take the same Markdown content, but this time put it into a preprocessor source file called article.md.pp.
"article.md.pp"
#lang pollen
Deep Thought
============
I am **so** happy to be writing this.
When you run this file in DrRacket, you’ll see:
Deep Thought ============
I am **so** happy to be writing this.
Hopefully, this result makes sense to you: when you run Markdown source in preprocessor mode, you get Markdown. When you run Markdown source in authoring mode, you get an X-expression.
6.4Templates
So how do you convert an X-expression into a finished file? You use a Pollen template, which takes data from an X-expression and converts it to the target format.
If you’ve used other web-publishing systems, templates are probably a familiar idea. Templates in Pollen are in some ways similar to the ones you’ve seen before, but in other ways different.
First, the similarities. At its simplest, a template holds boilerplate material that you want to reuse across multiple pages. For instance, in a set of HTML pages, you might have layout and navigation elements that stay the same, while the content changes. In that case, you could put the layout and navigation in the template, and keep the content in your Pollen source files. When you want to add a new page, you can make a new source file and just use it with the existing template. Moreover, if you want to change the layout and navigation globally, you can just change the template, rather than changing the source files.
Pollen templates, like others, can also have conditional features — meaning, you can embed simple code in your templates that allows them to change based on the content in the page. For instance, a template could show or hide “previous page” and “next page” links depending on whether there’s actually a previous or next page.
The major difference with Pollen templates is that there’s no special “template language” you need to learn, with magic delimiters and whatnot. Instead, you can use all the same Pollen commands in a template that you can in authoring mode or preprocessor mode.
To see a template in action, let’s return to the source file we started in the last section:
"article.html.pmd"
#lang pollen
Deep Thought
============
I am **so** happy to be writing this.
Last time, I had you run this file in DrRacket to see the X-expression it produced. This time, load it in the project server. You’ll see something like this:
Deep Thought
I am so happy to be writing this.
Here, you’re seeing the X-expression from your source combined with an HTML template, which adds the necessary boilerplate for the finished HTML:
<html><head><meta charset="UTF-8" /></head><body>
<root><h1 id="my-article">Deep Thought</h1><p>I am
<strong>so</strong> happy to be writing this.</p></root>
@@ -7,4 +7,4 @@
(strong () "so") " happy to be writing this."))
Either way, now we know that the text Deep Thought lives in the h1 tag. So we update our template accordingly (for brevity, I’m going to omit the style tag in these examples, but it’s fine to leave it in):
"template.html"
<html>
<head><meta charset="UTF-8">
<title>◊(select 'h1 doc)</title></head>
<body>◊(->html doc)</body>
</html>
When you refresh the page in the project server, the page title will now appear as “Deep Thought.” Of course, you can also combine static and dynamic elements in your template, like so:
"template.html"
<html>
<head><meta charset="UTF-8">
<title>◊(select 'h1 doc), by MB</title></head>
<body>◊(->html doc)</body>
</html>
The page title will now be “Deep Thought, by MB”.
A couple notes on command syntax. We inserted the select and ->html commands using Racket-mode syntax. We could also use text-mode syntax and write the commands this way:
"template.html"
<html>
<head><meta charset="UTF-8">
<title>◊select['h1 doc], by MB</title></head>
<body>◊->html[doc]</body>
</html>
This is exactly equivalent to the previous example. Skeptics are welcome to confirm this by checking the result in the project server.
Finally, notice that in the select command, the tag name 'h1 is written with a quote mark, whereas doc is not. This is an easy place to get tripped up, but the rule is simple: you don’t use a quote mark when you’re referring to the name of an existing function or variable (like select or doc). But you do need a quote mark when you’re using the text as a literal value.
Racket (and hence Pollen) makes a distinction between Symbols (e.g. 'h1) and Strings (e.g. "h1"). Without getting into the weeds, just note for now that the tag of an X-expression is always a symbol, not a string. But if you write ◊(select "h1" doc), the command will still work, because Pollen will treat it as ◊(select 'h1 doc), consistent with a general policy of not being persnickety about input types when the intention is clear.
6.4.4Linking to an external CSS file
If you’re a super web hotshot, you probably don’t put your CSS selectors in the <head> tag. Instead, you link to an external CSS file. So it will not surprise you that in Pollen, you can do this by adding the usual <link> tag to your HTML template, in this case a file called styles.css:
Fans of hand-coded CSS, I trust you to take it from here: make your styles.css file, and enjoy the results.
But folks who paid attention during the First tutorial might be wondering “Can we link to a dynamically generated styles.css.pp file?”
Yes, of course. Here’s the rule of thumb: when you’re making links between files — whether CSS, or HTML, or anything else — Pollen doesn’t care whether the file is static or dynamic. You just refer to it by its ultimate name, in this case styles.css. If a static styles.css file exists, Pollen will use that. If it doesn’t, Pollen will look for a source file it can use to make styles.css, and generate it on the spot. (You can also start with a static file, and change it to be dynamic later, and Pollen will do the right thing.)
So to use a dynamic CSS file, we don’t need to make any changes to template.html. We just need to add styles.css.pp to the project directory:
"styles.css.pp"
#lang pollen
◊(define h1-color "blue")
◊(define strong-color "green")
body {padding: 3em; font-size: 20px;}
h1 {background: ◊|h1-color|; color: white;}
strong {color: ◊|strong-color|;}
Now, when you refresh article.html in the project server, Pollen will generate the styles.css file it needs, and you’ll see the new colors in the page. As before, if you update styles.css.pp, Pollen will notice and regenerate the CSS file when you refresh the page.
Can you add multiple dynamic style sheets? Yes.
Can you mix dynamic and static style sheets? Yes.
Can you add a dynamic JavaScript file? Yes.
- You’re getting the general idea, right? So let’s move on.
6.5Intermission
If you only need one page for your article, you can stop here. You now know everything necessary to publish a single-page article using authoring mode. You know how to create the mandatory ingredients — a source file and a template — and you also know how to link to an optional CSS file, which can be dynamically generated.
If you want to create a multi-page article, however, you need to get through one more big idea. This might be a good time to take a break.
6.6Pagetrees
A pagetree is a hierarchical list of Pollen pages. When you have multiple pages in your project, the pagetree establishes relationships among those pages. At its most basic, a pagetree establishes a linear sequence for the pages. But pagetrees can also establish hierarchical relationships — for instance, a book-length project can be organized into chapters, the chapters into sections, and so on. The pagetree doesn’t impose any semantics on the organization of your project. It’s just a tree, and it’s up to you how many layers to establish, what those layers mean, and so on.
Pagemap might’ve been an equally good name, and perhaps more consistent with similar concepts in other web-publishing systems. But I avoided it out of deference to Racket’s map function, which means something completely different.
6.6.1Pagetree navigation
Pagetrees are used in various ways throughout Pollen. But the most obvious use for a pagetree is to add navigational links to your pages. Obviously, in a multi-page article, readers need a way of getting from one page to the next. In this part of the tutorial, we’ll expand our sample article from one page to three, and see how to create “previous page” and “next page” links in our template that are dynamically generated relative to the current page.
6.6.2Using the automatic pagetree
You’ve actually already been exposed to pagetrees (though I didn’t tell you about it at the time). Recall that the dashboard of the project server is located at http://localhost:8080/index.ptree. The list of files you see in the dashboard is a pagetree that Pollen creates by reading the files in the current directory and arranging them in alphabetical order.
If the multiple pages in your project are already ordered by filename, then you can rely on this automatic pagetree.
From earlier in the tutorial, you have a Markdown source file called article.html.pmd that looks like this:
"article.html.pmd"
#lang pollen
Deep Thought
============
I am **so** happy to be writing this.
Let’s supplement this source file by creating two others for the project:
"barticle.html.pmd"
#lang pollen
Barticle Title
==============
The wonderful second part of the article.
"carticle.html.pmd"
#lang pollen
Carticle Title
==============
The terrific third part.
As before, you can fill these source files with any sample Markdown content you like. Moreover, you don’t have to use the filenames barticle.html.pmd and carticle.html.pmd — the point is that the intended sequence needs to match the alphabetic sorting of the filenames.
We’ll reuse the template.html and styles.css files from earlier in the tutorial. Move or delete the other tutorial files so that your dashboard in the project server shows only these five files:
article.html.pmd
barticle.html.pmd
carticle.html.pmd
styles.css (or styles.css.pp)
template.html
If you click on any of the three Markdown sources, you will see it converted into HTML using template.html, with styles appiled from styles.css.
The automatic pagetree for this project is exactly what you see in the dashboard: a list of the three article files, followed by styles.css and template.html.
6.6.3Adding navigation links to the template with here
Recall from earlier in the tutorial that the content of your source file is made available in the template through the special variable doc. Likewise, the name of the current source file is made available through the special variable here.
To make any navigation link — up, down, sideways — the general idea is that we use here as input to a pagetree-navigation function, which then looks up the answer in the current pagetree.
First, let’s just see here on its own. Update your template as follows:
If you refresh article.html, you will now see the line “The current page is called article.html.” Switch to barticle.html, and you’ll see “The current page is called barticle.html.” Makes sense, right?
Now let’s use pagetree functions to show the names of the previous and next pages. Consistent with the usual policy of obviousness, these functions are called previous and next:
Refresh barticle.html. You’ll now see that “The current page is called barticle.html. The previous is article.html. The next is carticle.html.” So far, so good: we’re correctly deriving the previous and next pages from the automatic pagetree.
All that’s left is to add hyperlinks, which is easy:
The previous is <a href="◊|(previous here)|">◊|(previous here)|</a>.
The next is <a href="◊|(next here)|">◊|(next here)|</a>.
</body>
</html>
Refresh barticle.html, and you’ll see that the names of the previous and next pages are now hyperlinks to those pages. Click through and convince yourself that it works.
The documentation for pagetree Navigation will tell you about the other functions available for generating navigation links.
6.6.4Handling navigation boundaries with conditionals
If you clicked through to article.html or carticle.html, you might’ve noticed a couple problems. Because article.html is the first page in the automatic pagetree, it doesn’t have any previous page it can link to. And the next-page link for carticle.html is styles.css, which is strictly correct — it is, in fact, the next file in the automatic pagetree — but it’s not part of our article, so we’d rather stop the navigation there.
One way to fix the problem would be to have three separate template files — the standard one with both previous- and next-page links, one with only a next-page link, and one with only a previous-page link.
But since we have a whole programming language available in Pollen, that’s a dull-witted way to solve the problem. The better way is to add conditionals to the template to selectively change the navigation. That keeps things simple, because we’ll still have only one template.html to deal with.
To handle article.html, we want to hide the previous-page navigation link when there’s no previous page. As it turns out, if the previous function can’t find a previous page, it will return false. So we just need to wrap our previous-page navigation in the when/block command like so:
◊when/block[(previous here)]{The previous is <a href="◊|(previous here)|">◊|(previous here)|</a>.}
The next is <a href="◊|(next here)|">◊|(next here)|</a>.
</body>
</html>
The basic structure of when/block is ◊when/block[condition]{insert-this-text}. Note the square braces around the condition, and the curly braces around the text. Using (previous here) as the condition is shorthand for “when (previous here) does not return false...”
Programmers in the audience might be getting anxious about the repeated use of (previous here) — you’re welcome to store that value in a variable, and everything will work the same way:
◊when/block[prev-page]{The previous is <a href="◊|prev-page|">◊|prev-page|</a>.}
The next is <a href="◊|(next here)|">◊|(next here)|</a>.
</body>
</html>
We need a different technique for handling the end of the next-page navigation, because we’re not reaching the actual end of the pagetree. We’re just reaching the end of the pages we care about navigating through.
What condition will help us detect this? Here, we can notice that the names of our article pages all contain the string article. While you’d probably want a more robust condition for a real project, in this tutorial, what we’ll do is hide the next-page navigation if the name of the next page doesn’t contain “article”. As we did before, we wrap our navigation line in the when/block function:
The next is <a href="◊|(next here)|">◊|(next here)|</a>.}
</body>
</html>
This time, the condition is (regexp-match"article"(->string(nexthere))). How were you supposed to know this? You weren’t. That’s why this is a tutorial. Without going on a lengthy detour, the regexp-match function returns true if the first string (in this case, "article") is found inside the second string (in this case, we convert (nexthere) to a string by wrapping it in ->string).
In any case, even if some of the programmy bits went over your head just now, relax and paste the code into your template. What you’ll see when you refresh carticle.html is that the next-page link is gone. So now our template lets us navigate among the pages of our article, and the conditionals handle the end pages correctly.
6.6.5Making a pagetree file
I didn’t want to dwell on programming complications in the last conditional. Why? The extra programming was necessary only because we made life somewhat difficult for ourselves by relying on the automatic pagetree. A better way to solve the problem is to avoid it altogether by making a pagetree file.
Pagetree source files have a different syntax and status than other Pollen source files, so they are parsed using their own Pollen dialect. To invoke this dialect, you just start the file with #lang pollen and name the file with the ptree extension, for instance my-project.ptree. While you can have as many pagetrees in your project as you want, Pollen will accord primary status to the one named index.ptree.
So let’s make an index.ptree file. At its simplest, a pagetree file can just be a list of files in the intended order. In DrRacket, create a new file in your project directory as follows:
Pretty boring, I know. But behind the scenes, Pollen’s pagetree parser is making sure your tree is valid (e.g., no duplicate or malformed names). Today it’s boring, but on the day you have a long and complicated pagetree, you will be grateful.
Notice that the names in this pagetree are the names of output files, not source files. This is deliberate, so that neither you nor Pollen has to care which files are static vs. dynamic. This next pagetree wouldn’t be wrong in the sense of bad syntax — the pagetree parser won’t complain — but it would be wrong in the sense of not-what-you-want, because it refers to source names rather than output names:
"bad-index.ptree"
#lang pollen
carticle.html.pmd
article.html.pmd
barticle.html.pmd
You also probably noticed that the files are in a different order than they were in the automatic pagetree: carticle.html is first, followed by article.html and then barticle.html. This too is deliberate, so we can see what happens with a differently ordered pagetree.
Pagetrees don’t change nearly as often as other source files, so as a performance optimization, the project server does not dynamically reflect changes to pagetrees. To see the effect of this new pagetree on our project, you’ll need to go to your terminal window and stop the project server with ctrl+C, and then restart it. Which will take all of three seconds.
Now refresh carticle.html. You’ll notice that the navigation links are different. You won’t see a previous-page link — because carticle.html is now the first page in the pagetree — and the next page will show up as article.html. Click through to article.html, and you’ll see the navigation likewise updated. Click through to barticle.html, and you’ll see ...
BAM! An error page with a yellow box that says Can’t convert #f to string. What happened? We switched to using our own pagetree file but we didn’t update our template conditionals. Once you reach barticle.html, the value of (nexthere) is false, which means the (->string(nexthere)) command in the template conditional is trying to convert false into a string. Hence the error.
So let’s go back and fix that. Because we don’t have extraneous files in our pagetree anymore, we can change the second conditional in the template to work the same way as the first:
◊when/block[prev-page]{The previous is <a href="◊|prev-page|">◊|prev-page|</a>.}
◊(define next-page (next here))
◊when/block[next-page]{
The next is <a href="◊|next-page|">◊|next-page|</a>.}
</body>
</html>
Refresh barticle.html — because you’re updating the template, you don’t need to restart the project server — and you’ll see the right result. The previous-page link goes to article.html, and the next-page link is hidden.
6.6.6index.ptree & the project server
One more thing to show you before we wrap up this tutorial. Remember that the dashboard of the project server is at http://localhost:8080/index.ptree? By default, the project server will synthesize a pagetree from an alphbetical directory listing.
But if you put your own index.ptree file in the directory, the project server will use that for the dashboard instead. In fact, visit http://localhost:8080/index.ptree now and you’ll see what I mean. Consistent with the index.ptree you made, you’ll now see carticle.html, article.html, and barticle.html, but not template.html nor styles.css (even though they’re still in the project directory).
6.7Second tutorial complete
That was a big tutorial. I commend you for your tenacity and patience. But in this tutorial, you made a giant leap forward. Despite the silly examples, you now know everything you need to make multi-page articles — books, even — using Markdown authoring mode in Pollen. If this is all you ever use Pollen for, it’ll be a big improvement over ordinary Markdown.
But there’s more. We haven’t even gotten into the more elaborate automation that’s possible with Pollen, nor Pollen’s own markup language. We’ll cover that in the third tutorial.
\ No newline at end of file
+ You’re getting the general idea, right? So let’s move on.
6.5Intermission
If you only need one page for your article, you can stop here. You now know everything necessary to publish a single-page article using authoring mode. You know how to create the mandatory ingredients — a source file and a template — and you also know how to link to an optional CSS file, which can be dynamically generated.
If you want to create a multi-page article, however, you need to get through one more big idea. This might be a good time to take a break.
6.6Pagetrees
A pagetree is a hierarchical list of Pollen pages. When you have multiple pages in your project, the pagetree establishes relationships among those pages. At its most basic, a pagetree establishes a linear sequence for the pages. But pagetrees can also establish hierarchical relationships — for instance, a book-length project can be organized into chapters, the chapters into sections, and so on. The pagetree doesn’t impose any semantics on the organization of your project. It’s just a tree, and it’s up to you how many layers to establish, what those layers mean, and so on.
Pagemap might’ve been an equally good name, and perhaps more consistent with similar concepts in other web-publishing systems. But I avoided it out of deference to Racket’s map function, which means something completely different.
6.6.1Pagetree navigation
Pagetrees are used in various ways throughout Pollen. But the most obvious use for a pagetree is to add navigational links to your pages. Obviously, in a multi-page article, readers need a way of getting from one page to the next. In this part of the tutorial, we’ll expand our sample article from one page to three, and see how to create “previous page” and “next page” links in our template that are dynamically generated relative to the current page.
6.6.2Using the automatic pagetree
You’ve actually already been exposed to pagetrees (though I didn’t tell you about it at the time). Recall that the dashboard of the project server is located at http://localhost:8080/index.ptree. The list of files you see in the dashboard is a pagetree that Pollen creates by reading the files in the current directory and arranging them in alphabetical order.
If the multiple pages in your project are already ordered by filename, then you can rely on this automatic pagetree.
From earlier in the tutorial, you have a Markdown source file called article.html.pmd that looks like this:
"article.html.pmd"
#lang pollen
Deep Thought
============
I am **so** happy to be writing this.
Let’s supplement this source file by creating two others for the project:
"barticle.html.pmd"
#lang pollen
Barticle Title
==============
The wonderful second part of the article.
"carticle.html.pmd"
#lang pollen
Carticle Title
==============
The terrific third part.
As before, you can fill these source files with any sample Markdown content you like. Moreover, you don’t have to use the filenames barticle.html.pmd and carticle.html.pmd — the point is that the intended sequence needs to match the alphabetic sorting of the filenames.
We’ll reuse the template.html and styles.css files from earlier in the tutorial. Move or delete the other tutorial files so that your dashboard in the project server shows only these five files:
article.html.pmd
barticle.html.pmd
carticle.html.pmd
styles.css (or styles.css.pp)
template.html
If you click on any of the three Markdown sources, you will see it converted into HTML using template.html, with styles appiled from styles.css.
The automatic pagetree for this project is exactly what you see in the dashboard: a list of the three article files, followed by styles.css and template.html.
6.6.3Adding navigation links to the template with here
Recall from earlier in the tutorial that the content of your source file is made available in the template through the special variable doc. Likewise, the name of the current source file is made available through the special variable here.
To make any navigation link — up, down, sideways — the general idea is that we use here as input to a pagetree-navigation function, which then looks up the answer in the current pagetree.
First, let’s just see here on its own. Update your template as follows:
If you refresh article.html, you will now see the line “The current page is called article.html.” Switch to barticle.html, and you’ll see “The current page is called barticle.html.” Makes sense, right?
Now let’s use pagetree functions to show the names of the previous and next pages. Consistent with the usual policy of obviousness, these functions are called previous and next:
Refresh barticle.html. You’ll now see that “The current page is called barticle.html. The previous is article.html. The next is carticle.html.” So far, so good: we’re correctly deriving the previous and next pages from the automatic pagetree.
All that’s left is to add hyperlinks, which is easy:
The previous is <a href="◊|(previous here)|">◊|(previous here)|</a>.
The next is <a href="◊|(next here)|">◊|(next here)|</a>.
</body>
</html>
Refresh barticle.html, and you’ll see that the names of the previous and next pages are now hyperlinks to those pages. Click through and convince yourself that it works.
The documentation for pagetree Navigation will tell you about the other functions available for generating navigation links.
6.6.4Handling navigation boundaries with conditionals
If you clicked through to article.html or carticle.html, you might’ve noticed a couple problems. Because article.html is the first page in the automatic pagetree, it doesn’t have any previous page it can link to. And the next-page link for carticle.html is styles.css, which is strictly correct — it is, in fact, the next file in the automatic pagetree — but it’s not part of our article, so we’d rather stop the navigation there.
One way to fix the problem would be to have three separate template files — the standard one with both previous- and next-page links, one with only a next-page link, and one with only a previous-page link.
But since we have a whole programming language available in Pollen, that’s a dull-witted way to solve the problem. The better way is to add conditionals to the template to selectively change the navigation. That keeps things simple, because we’ll still have only one template.html to deal with.
To handle article.html, we want to hide the previous-page navigation link when there’s no previous page. As it turns out, if the previous function can’t find a previous page, it will return false. So we just need to wrap our previous-page navigation in the when/block command like so:
◊when/block[(previous here)]{The previous is <a href="◊|(previous here)|">◊|(previous here)|</a>.}
The next is <a href="◊|(next here)|">◊|(next here)|</a>.
</body>
</html>
The basic structure of when/block is ◊when/block[condition]{insert-this-text}. Note the square braces around the condition, and the curly braces around the text. Using (previous here) as the condition is shorthand for “when (previous here) does not return false...”
Programmers in the audience might be getting anxious about the repeated use of (previous here) — you’re welcome to store that value in a variable, and everything will work the same way:
◊when/block[prev-page]{The previous is <a href="◊|prev-page|">◊|prev-page|</a>.}
The next is <a href="◊|(next here)|">◊|(next here)|</a>.
</body>
</html>
We need a different technique for handling the end of the next-page navigation, because we’re not reaching the actual end of the pagetree. We’re just reaching the end of the pages we care about navigating through.
What condition will help us detect this? Here, we can notice that the names of our article pages all contain the string article. While you’d probably want a more robust condition for a real project, in this tutorial, what we’ll do is hide the next-page navigation if the name of the next page doesn’t contain “article”. As we did before, we wrap our navigation line in the when/block function:
The next is <a href="◊|(next here)|">◊|(next here)|</a>.}
</body>
</html>
This time, the condition is (regexp-match"article"(->string(nexthere))). How were you supposed to know this? You weren’t. That’s why this is a tutorial. Without going on a lengthy detour, the regexp-match function returns true if the first string (in this case, "article") is found inside the second string (in this case, we convert (nexthere) to a string by wrapping it in ->string).
In any case, even if some of the programmy bits went over your head just now, relax and paste the code into your template. What you’ll see when you refresh carticle.html is that the next-page link is gone. So now our template lets us navigate among the pages of our article, and the conditionals handle the end pages correctly.
6.6.5Making a pagetree file
I didn’t want to dwell on programming complications in the last conditional. Why? The extra programming was necessary only because we made life somewhat difficult for ourselves by relying on the automatic pagetree. A better way to solve the problem is to avoid it altogether by making a pagetree file.
Pagetree source files have a different syntax and status than other Pollen source files, so they are parsed using their own Pollen dialect. To invoke this dialect, you just start the file with #lang pollen and name the file with the ptree extension, for instance my-project.ptree. While you can have as many pagetrees in your project as you want, Pollen will accord primary status to the one named index.ptree.
So let’s make an index.ptree file. At its simplest, a pagetree file can just be a list of files in the intended order. In DrRacket, create a new file in your project directory as follows:
Pretty boring, I know. But behind the scenes, Pollen’s pagetree parser is making sure your tree is valid (e.g., no duplicate or malformed names). Today it’s boring, but on the day you have a long and complicated pagetree, you will be grateful.
Notice that the names in this pagetree are the names of output files, not source files. This is deliberate, so that neither you nor Pollen has to care which files are static vs. dynamic. This next pagetree wouldn’t be wrong in the sense of bad syntax — the pagetree parser won’t complain — but it would be wrong in the sense of not-what-you-want, because it refers to source names rather than output names:
"bad-index.ptree"
#lang pollen
carticle.html.pmd
article.html.pmd
barticle.html.pmd
You also probably noticed that the files are in a different order than they were in the automatic pagetree: carticle.html is first, followed by article.html and then barticle.html. This too is deliberate, so we can see what happens with a differently ordered pagetree.
Pagetrees don’t change nearly as often as other source files, so as a performance optimization, the project server does not dynamically reflect changes to pagetrees. To see the effect of this new pagetree on our project, you’ll need to go to your terminal window and stop the project server with ctrl+C, and then restart it. Which will take all of three seconds.
Now refresh carticle.html. You’ll notice that the navigation links are different. You won’t see a previous-page link — because carticle.html is now the first page in the pagetree — and the next page will show up as article.html. Click through to article.html, and you’ll see the navigation likewise updated. Click through to barticle.html, and you’ll see ...
BAM! An error page with a yellow box that says Can’t convert #f to string. What happened? We switched to using our own pagetree file but we didn’t update our template conditionals. Once you reach barticle.html, the value of (nexthere) is false, which means the (->string(nexthere)) command in the template conditional is trying to convert false into a string. Hence the error.
So let’s go back and fix that. Because we don’t have extraneous files in our pagetree anymore, we can change the second conditional in the template to work the same way as the first:
◊when/block[prev-page]{The previous is <a href="◊|prev-page|">◊|prev-page|</a>.}
◊(define next-page (next here))
◊when/block[next-page]{
The next is <a href="◊|next-page|">◊|next-page|</a>.}
</body>
</html>
Refresh barticle.html — because you’re updating the template, you don’t need to restart the project server — and you’ll see the right result. The previous-page link goes to article.html, and the next-page link is hidden.
6.6.6index.ptree & the project server
One more thing to show you before we wrap up this tutorial. Remember that the dashboard of the project server is at http://localhost:8080/index.ptree? By default, the project server will synthesize a pagetree from an alphbetical directory listing.
But if you put your own index.ptree file in the directory, the project server will use that for the dashboard instead. In fact, visit http://localhost:8080/index.ptree now and you’ll see what I mean. Consistent with the index.ptree you made, you’ll now see carticle.html, article.html, and barticle.html, but not template.html nor styles.css (even though they’re still in the project directory).
6.7Second tutorial complete
That was a big tutorial. I commend you for your tenacity and patience. But in this tutorial, you made a giant leap forward. Despite the silly examples, you now know everything you need to make multi-page articles — books, even — using Markdown authoring mode in Pollen. If this is all you ever use Pollen for, it’ll be a big improvement over ordinary Markdown.
But there’s more. We haven’t even gotten into the more elaborate automation that’s possible with Pollen, nor Pollen’s own markup language. We’ll cover that in the third tutorial.
\ No newline at end of file
diff --git a/scribblings/decode.scrbl b/scribblings/decode.scrbl
index 4e595f0..26e68b9 100644
--- a/scribblings/decode.scrbl
+++ b/scribblings/decode.scrbl
@@ -309,7 +309,9 @@ Within @racket[_tagged-xexpr-elements], convert occurrences of @racket[_linebrea
[#:tag paragraph-tag symbol? 'p]
[#:linebreak-proc linebreak-proc (txexpr-elements? . -> . txexpr-elements?) detect-linebreaks])
txexpr-elements?]
-Find paragraphs within @racket[_elements], as denoted by @racket[_paragraph-sep], and wrap them with @racket[_paragraph-tag], unless the @racket[_element] is already a @racket[block-txexpr?] (because in that case, the wrapping is superfluous). Thus, as a consequence, if @racket[_paragraph-sep] occurs between two blocks, it's ignored.
+Find paragraphs within @racket[_elements] (as denoted by @racket[_paragraph-sep]) and wrap them with @racket[_paragraph-tag]. Also handle linebreaks using @racket[detect-linebreaks].
+
+If @racket[_element] is already a @racket[block-txexpr?], it will not be wrapped as a paragraph (because in that case, the wrapping would be superfluous). Thus, as a consequence, if @racket[_paragraph-sep] occurs between two blocks, it will be ignored (as in the example below using two sequential @racket['div] blocks.)
The @racket[_paragraph-tag] argument sets the tag used to wrap paragraphs.