From 9c258d718531f80600e2baacc15c01969a69d75a Mon Sep 17 00:00:00 2001 From: Matthew Butterick Date: Wed, 10 Sep 2014 18:31:06 -0700 Subject: [PATCH] updates to rendered docs --- doc/___219484.html | 2 ++ doc/doc-index.html | 2 +- doc/index.html | 2 +- doc/quick-tour.html | 2 +- doc/second-tutorial.html | 4 ++-- doc/third-tutorial.html | 10 ++-------- 6 files changed, 9 insertions(+), 13 deletions(-) create mode 100644 doc/___219484.html diff --git a/doc/___219484.html b/doc/___219484.html new file mode 100644 index 0000000..c91f554 --- /dev/null +++ b/doc/___219484.html @@ -0,0 +1,2 @@ + +
▼Pollen:   the book is a program
6.1.0.5
    ← prev  up  next → 

7 

    ← prev  up  next → 
 
\ No newline at end of file diff --git a/doc/doc-index.html b/doc/doc-index.html index dd76b4b..d668f7d 100644 --- a/doc/doc-index.html +++ b/doc/doc-index.html @@ -1,2 +1,2 @@ -Index
▼Pollen:   the book is a program
    ← prev  up  next → 

Index

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

 

“But I really need XML…”
“Now you have two problems”
#%top
->html
->markup-source-path
->null-source-path
->output-path
->pagenode
->preproc-source-path
->scribble-source-path
->template-source-path
A special data structure for HTML
Acknowledgments
Adding commands
Adding navigation links to the template with here
Any command is valid
Attaching behavior to tags
Attributes
Authoring mode
Backstory
Block
block-txexpr?
Cache
cache-ref
cached-require
children
Choosing custom tags
Command syntax using ◊
Creating a Pollen markup file
Creating a source file
Creating valid HTML output
current-cache
current-pagetree
Custom exports
Decode
decode
decode-elements
Decoding markup via the root tag
def/c
Defining variables with commands
detect-linebreaks
detect-paragraphs
Development environment
Enter Racket
File
File formats
First tutorial
First tutorial complete
Format independence
Functions
Further reading
get-template-for
Handling navigation boundaries with conditionals
has-markup-source?
has-null-source?
has-preproc-source?
has-scribble-source?
has-template-source?
has/is-markup-source?
has/is-null-source?
has/is-preproc-source?
has/is-scribble-source?
has/is-template-source?
in-pagetree?
index.ptree & the project server
Inserting a comment
Inserting metas
Inserting specific source data into templates
Inserting the value of a variable
Inserting values from variables
Inserting variables within CSS
Installation
Intermission
Intermission
Intermission
Interpolating variables into strings
Invoking other functions
Invoking tag functions
License & source code
Linking to an external CSS file
make-cache
make-default-tag-function
Making a custom template
Making a pagetree file
Making pagetrees by hand
Making pagetrees with a source file
Making sure raco pollen works
Markdown (.pmd extension)
Markdown authoring mode
Markdown in Pollen: two options
Markdown mode
Markup (.pm extension)
Markup mode
markup-source?
Module reference
Multiple input values & rest arguments
Naming, saving, and rendering a source file
Navigation
next
next*
Notes for experienced programmers
Null (.p extension)
null-source?
One language, multiple dialects
Organizing functions
pagenode?
pagenodeish?
Pagetree
Pagetree (.ptree extension)
Pagetree navigation
pagetree->list
pagetree-source?
pagetree?
Pagetrees
Pagetrees
parent
Parsing attributes
path->pagenode
Point of no return
pollen
Pollen as a preprocessor
Pollen command syntax
Pollen markup vs. XML
pollen/cache
pollen/decode
pollen/file
pollen/markdown
pollen/markup
pollen/pagetree
pollen/pre
pollen/ptree
pollen/render
pollen/tag
pollen/template
pollen/top
pollen/world
Pollen: the book is a program
Predicates & validation
Prelude: my principled objection to Markdown
preproc-source?
Preprocessor (.pp extension)
Prerequisites
Prerequisites
Prerequisites
previous
previous*
project-block-tags
PS for Scribble users
Putting in the text of the poem
Putting it all together
Quick tour
Racket basics (if you’re not familiar)
raco pollen
raco pollen clone
raco pollen help
raco pollen render
raco pollen start
register-block-tag
Render
render
render-batch
render-pagetree
render-to-file
render-to-file-if-needed
reset-cache
Rethinking the solution for digital books
Returning an X-expression
Running a source file
Saving & naming your source file
Scribble (.scrbl extension)
scribble-source?
Second tutorial
Second tutorial complete
select
select*
select-from-doc
select-from-metas
Semantic markup
Setting the #lang line
Setting up a preprocessor source file
siblings
smart-dashes
smart-quotes
Source files in the dashboard
Source formats
split-attributes
Standard exports
Starting a new file in DrRacket
Starting the project server with raco pollen
Tag
Tags & tag functions
Tags are functions
Template
template-source?
Templated source files
Templates
Templates
The ->html function and the doc variable
The better idea: a programming model
The big picture
The book is a program
The command name
The directory-require.rkt file
The directory-require.rkt file
The end of the beginning
The golden rule
The lozenge glyph (◊)
The preprocessor
The project server
The Racket arguments
The relationship of Racket & Pollen
The text argument
The two command modes: text mode & Racket mode
The XML problem
Third tutorial
Top
Typography
Using custom tags
Using index.ptree in the dashboard
Using Markdown with the preprocessor
Using pagetrees for navigation
Using pagetrees with raco pollen render
Using Racket’s function libraries
Using raco pollen
Using the automatic pagetree
Using the dashboard
Using the project server
Utilities
Utility formats
validate-pagetree
Web development and its discontents
What are custom tags good for?
What is Pollen?
What Pollen markup does differently
when/block
whitespace/nbsp?
whitespace?
Working with the preprocessor
World
world:check-directory-requires-in-render?
world:command-marker
world:current-server-extras-path
world:current-server-port
world:dashboard-css
world:decodable-extensions
world:default-pagetree
world:default-port
world:default-template-prefix
world:directory-require
world:fallback-template-prefix
world:linebreak-separator
world:main-pollen-export
world:markdown-source-ext
world:markup-source-ext
world:meta-pollen-export
world:mode-auto
world:mode-markdown
world:mode-markup
world:mode-pagetree
world:mode-preproc
world:newline
world:null-source-ext
world:pagetree-root-node
world:pagetree-source-ext
world:paragraph-separator
world:paths-excluded-from-dashboard
world:preproc-source-ext
world:scribble-source-ext
world:server-extras-dir
world:template-meta-key
world:template-source-ext
wrap-hanging-quotes
Writing with Pollen markup
X-expressions
◊ command overview

    ← prev  up  next → 
 
\ No newline at end of file +Index
▼Pollen:   the book is a program
    ← prev  up  next → 

Index

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

 

“But I really need XML…”
“Now you have two problems”
#%top
->html
->markup-source-path
->null-source-path
->output-path
->pagenode
->preproc-source-path
->scribble-source-path
->template-source-path
A CSS stylesheet using the preprocessor
A special data structure for HTML
Acknowledgments
Adding commands
Adding navigation links to the template with here
Any command is valid
Attaching behavior to tags
Attributes
Authoring mode
Backstory
Block
block-txexpr?
Cache
cache-ref
cached-require
children
Choosing custom tags
Command syntax using ◊
Creating a Pollen markup file
Creating a source file
Creating valid HTML output
current-cache
current-pagetree
Custom exports
Decode
decode
decode-elements
Decoding markup via the root tag
def/c
Defining variables with commands
detect-linebreaks
detect-paragraphs
Development environment
Enter Racket
File
File formats
First tutorial
First tutorial complete
Format independence
Functions
Further reading
get-template-for
Handling navigation boundaries with conditionals
has-markup-source?
has-null-source?
has-preproc-source?
has-scribble-source?
has-template-source?
has/is-markup-source?
has/is-null-source?
has/is-preproc-source?
has/is-scribble-source?
has/is-template-source?
in-pagetree?
index.ptree & the project server
Inserting a comment
Inserting metas
Inserting specific source data into templates
Inserting the value of a variable
Inserting values from variables
Inserting variables within CSS
Installation
Intermission
Intermission
Intermission
Interpolating variables into strings
Invoking other functions
Invoking tag functions
License & source code
Linking to an external CSS file
make-cache
make-default-tag-function
Making a custom template
Making a pagetree file
Making pagetrees by hand
Making pagetrees with a source file
Making sure raco pollen works
Markdown (.pmd extension)
Markdown authoring mode
Markdown in Pollen: two options
Markdown mode
Markup (.pm extension)
Markup mode
markup-source?
Module reference
Multiple input values & rest arguments
Naming, saving, and rendering a source file
Navigation
next
next*
Notes for experienced programmers
Null (.p extension)
null-source?
One language, multiple dialects
Organizing functions
pagenode?
pagenodeish?
Pagetree
Pagetree (.ptree extension)
Pagetree navigation
pagetree->list
pagetree-source?
pagetree?
Pagetrees
Pagetrees
parent
Parsing attributes
path->pagenode
Point of no return
pollen
Pollen as a preprocessor
Pollen command syntax
Pollen markup vs. XML
pollen/cache
pollen/decode
pollen/file
pollen/markdown
pollen/markup
pollen/pagetree
pollen/pre
pollen/ptree
pollen/render
pollen/tag
pollen/template
pollen/top
pollen/world
Pollen: the book is a program
Predicates & validation
Prelude: my principled objection to Markdown
preproc-source?
Preprocessor (.pp extension)
Prerequisites
Prerequisites
Prerequisites
previous
previous*
project-block-tags
PS for Scribble users
Putting in the text of the poem
Putting it all together
Quick tour
Racket basics (if you’re not familiar)
raco pollen
raco pollen clone
raco pollen help
raco pollen render
raco pollen start
register-block-tag
Render
render
render-batch
render-pagetree
render-to-file
render-to-file-if-needed
reset-cache
Rethinking the solution for digital books
Returning an X-expression
Running a source file
Saving & naming your source file
Scribble (.scrbl extension)
scribble-source?
Second tutorial
Second tutorial complete
select
select*
select-from-doc
select-from-metas
Semantic markup
Setting the #lang line
Setting up a preprocessor source file
siblings
smart-dashes
smart-quotes
Source files in the dashboard
Source formats
split-attributes
Standard exports
Starting a new file in DrRacket
Starting the project server with raco pollen
Tag
Tags & tag functions
Tags are functions
Template
template-source?
Templated source files
Templates
Templates
The ->html function and the doc variable
The better idea: a programming model
The big picture
The book is a program
The command name
The content source files using Pollen markup
The directory-require.rkt file
The directory-require.rkt file
The end of the beginning
The golden rule
The lozenge glyph (◊)
The pagetree
The preprocessor
The project server
The Racket arguments
The relationship of Racket & Pollen
The result
The template
The text argument
The two command modes: text mode & Racket mode
The XML problem
Third tutorial
Third tutorial complete
Top
Typography
Using custom tags
Using index.ptree in the dashboard
Using Markdown with the preprocessor
Using pagetrees for navigation
Using pagetrees with raco pollen render
Using Racket’s function libraries
Using raco pollen
Using the automatic pagetree
Using the dashboard
Using the directory-require.rkt file
Using the project server
Utilities
Utility formats
validate-pagetree
Web development and its discontents
What are custom tags good for?
What is Pollen?
What Pollen markup does differently
when/block
whitespace/nbsp?
whitespace?
Working with the preprocessor
World
world:check-directory-requires-in-render?
world:command-marker
world:current-server-extras-path
world:current-server-port
world:dashboard-css
world:decodable-extensions
world:default-pagetree
world:default-port
world:default-template-prefix
world:directory-require
world:fallback-template-prefix
world:linebreak-separator
world:main-pollen-export
world:markdown-source-ext
world:markup-source-ext
world:meta-pollen-export
world:mode-auto
world:mode-markdown
world:mode-markup
world:mode-pagetree
world:mode-preproc
world:newline
world:null-source-ext
world:pagetree-root-node
world:pagetree-source-ext
world:paragraph-separator
world:paths-excluded-from-dashboard
world:preproc-source-ext
world:scribble-source-ext
world:server-extras-dir
world:template-meta-key
world:template-source-ext
wrap-hanging-quotes
Writing with Pollen markup
X-expressions
◊ command overview

    ← prev  up  next → 
 
\ No newline at end of file diff --git a/doc/index.html b/doc/index.html index e9af041..5184aac 100644 --- a/doc/index.html +++ b/doc/index.html @@ -1,3 +1,3 @@ Pollen: the book is a program
▼Pollen:   the book is a program
On this page:
Pollen:   the book is a program
6.1.0.5
    ← prev  up  next → 

Pollen: the book is a program

Matthew Butterick <mb@mbtype.com>

 #lang pollen package: pollen

Pollen is a publishing system that helps authors create beautiful and functional web-based books. Pollen includes tools for writing, designing, programming, testing, and publishing.

I used Pollen to create my book Butterick’s Practical Typography. Sure, go take a look. Is it better than the last digital book you encountered? Yes it is. Would you like your book to look like that? If so, keep reading.

At the core of Pollen is an argument: -

  • First, that digital books should be the best books we’ve ever had. So far, they’re not even close.

  • Second, that because digital books are software, an author shouldn’t think of a book as merely data. The book is a program.

  • Third, that the way we make digital books better than their predecessors is by exploiting this programmability.

That’s what Pollen is for.

Not that you need to be a programmer to use Pollen. On the contrary, the Pollen language is markup-based, so you can write & edit text naturally. But when you want to automate repetitive tasks, add cross-references, or pull in data from other sources, you can access a full programming language from within the text.

That language is Racket. I chose Racket because while the idea for Pollen had been with me for several years, it simply wasn’t possible to build it with other languages. So if it’s unfamiliar to you, don’t panic. It was unfamiliar to me. Once you see what you can do with Pollen & Racket, you may be persuaded. I was.

Or, if you can find a better digital-publishing tool, use that. But I’m never going back to the way I used to work.

    1 Installation

    2 Quick tour

      2.1 Creating a source file

      2.2 Running a source file

      2.3 Naming, saving, and rendering a source file

      2.4 The project server

      2.5 Intermission

      2.6 Pollen as a preprocessor

      2.7 Markdown mode

      2.8 Markup mode

      2.9 Templates

      2.10 PS for Scribble users

      2.11 The end of the beginning

    3 Backstory

      3.1 Web development and its discontents

      3.2 The better idea: a programming model

      3.3 “Now you have two problems”

      3.4 Rethinking the solution for digital books

      3.5 Enter Racket

      3.6 What is Pollen?

    4 The big picture

      4.1 The book is a program

      4.2 One language, multiple dialects

      4.3 Development environment

      4.4 A special data structure for HTML

      4.5 Pollen command syntax

      4.6 The preprocessor

      4.7 Templated source files

      4.8 Pagetrees

    5 First tutorial

      5.1 Prerequisites

      5.2 The relationship of Racket & Pollen

      5.3 Starting a new file in DrRacket

        5.3.1 Setting the #lang line

        5.3.2 Putting in the text of the poem

        5.3.3 Saving & naming your source file

      5.4 Using the project server

        5.4.1 Starting the project server with raco pollen

        5.4.2 Using the dashboard

        5.4.3 Source files in the dashboard

      5.5 Working with the preprocessor

        5.5.1 Setting up a preprocessor source file

        5.5.2 Creating valid HTML output

        5.5.3 Adding commands

        5.5.4 Racket basics (if you’re not familiar)

        5.5.5 Defining variables with commands

        5.5.6 Inserting values from variables

        5.5.7 Inserting variables within CSS

      5.6 First tutorial complete

    6 Second tutorial

      6.1 Prerequisites

      6.2 Prelude: my principled objection to Markdown

      6.3 Markdown in Pollen: two options

        6.3.1 Using Markdown with the preprocessor

        6.3.2 Authoring mode

        6.3.3 X-expressions

        6.3.4 Markdown authoring mode

      6.4 Templates

        6.4.1 The ->html function and the doc variable

        6.4.2 Making a custom template

        6.4.3 Inserting specific source data into templates

        6.4.4 Linking to an external CSS file

      6.5 Intermission

      6.6 Pagetrees

        6.6.1 Pagetree navigation

        6.6.2 Using the automatic pagetree

        6.6.3 Adding navigation links to the template with here

        6.6.4 Handling navigation boundaries with conditionals

        6.6.5 Making a pagetree file

        6.6.6 index.ptree & the project server

      6.7 Second tutorial complete

    7 Third tutorial

      7.1 Prerequisites

      7.2 Pollen markup vs. XML

        7.2.1 The XML problem

        7.2.2 What Pollen markup does differently

        7.2.3 “But I really need XML…”

      7.3 Writing with Pollen markup

        7.3.1 Creating a Pollen markup file

        7.3.2 Tags & tag functions

        7.3.3 Attributes

        7.3.4 What are custom tags good for?

        7.3.5 Semantic markup

        7.3.6 Format independence

        7.3.7 Using custom tags

        7.3.8 Choosing custom tags

      7.4 Tags are functions

        7.4.1 Attaching behavior to tags

        7.4.2 Notes for experienced programmers

          7.4.2.1 Point of no return

          7.4.2.2 Multiple input values & rest arguments

          7.4.2.3 Returning an X-expression

          7.4.2.4 Interpolating variables into strings

          7.4.2.5 Parsing attributes

      7.5 Intermission

      7.6 Organizing functions

        7.6.1 Using Racket’s function libraries

        7.6.2 The directory-require.rkt file

      7.7 Decoding markup via the root tag

      7.8 Putting it all together

    8 Using raco pollen

      8.1 Making sure raco pollen works

      8.2 raco pollen

      8.3 raco pollen help

      8.4 raco pollen start

      8.5 raco pollen render

      8.6 raco pollen clone

    9 File formats

      9.1 Source formats

        9.1.1 Command syntax using ◊

        9.1.2 Any command is valid

        9.1.3 Standard exports

        9.1.4 Custom exports

        9.1.5 The directory-require.rkt file

        9.1.6 Preprocessor (.pp extension)

        9.1.7 Markdown (.pmd extension)

        9.1.8 Markup (.pm extension)

        9.1.9 Pagetree (.ptree extension)

      9.2 Utility formats

        9.2.1 Scribble (.scrbl extension)

        9.2.2 Null (.p extension)

    10 ◊ command overview

      10.1 The golden rule

      10.2 The lozenge glyph (◊)

      10.3 The two command modes: text mode & Racket mode

        10.3.1 The command name

          10.3.1.1 Invoking tag functions

          10.3.1.2 Invoking other functions

          10.3.1.3 Inserting the value of a variable

          10.3.1.4 Inserting metas

          10.3.1.5 Inserting a comment

        10.3.2 The Racket arguments

        10.3.3 The text argument

      10.4 Further reading

    11 Module reference

      11.1 Cache

      11.2 Decode

        11.2.1 Block

        11.2.2 Typography

      11.3 File

      11.4 Pagetree

        11.4.1 Making pagetrees with a source file

        11.4.2 Making pagetrees by hand

        11.4.3 Using pagetrees for navigation

        11.4.4 Using index.ptree in the dashboard

        11.4.5 Using pagetrees with raco pollen render

        11.4.6 Functions

          11.4.6.1 Predicates & validation

          11.4.6.2 Navigation

          11.4.6.3 Utilities

      11.5 Render

      11.6 Template

      11.7 Tag

      11.8 Top

      11.9 World

    12 Acknowledgments

    13 License & source code

    Index

    ← prev  up  next → 
 
\ No newline at end of file +

That’s what Pollen is for.

Not that you need to be a programmer to use Pollen. On the contrary, the Pollen language is markup-based, so you can write & edit text naturally. But when you want to automate repetitive tasks, add cross-references, or pull in data from other sources, you can access a full programming language from within the text.

That language is Racket. I chose Racket because while the idea for Pollen had been with me for several years, it simply wasn’t possible to build it with other languages. So if it’s unfamiliar to you, don’t panic. It was unfamiliar to me. Once you see what you can do with Pollen & Racket, you may be persuaded. I was.

Or, if you can find a better digital-publishing tool, use that. But I’m never going back to the way I used to work.

    1 Installation

    2 Quick tour

      2.1 Creating a source file

      2.2 Running a source file

      2.3 Naming, saving, and rendering a source file

      2.4 The project server

      2.5 Intermission

      2.6 Pollen as a preprocessor

      2.7 Markdown mode

      2.8 Markup mode

      2.9 Templates

      2.10 PS for Scribble users

      2.11 The end of the beginning

    3 Backstory

      3.1 Web development and its discontents

      3.2 The better idea: a programming model

      3.3 “Now you have two problems”

      3.4 Rethinking the solution for digital books

      3.5 Enter Racket

      3.6 What is Pollen?

    4 The big picture

      4.1 The book is a program

      4.2 One language, multiple dialects

      4.3 Development environment

      4.4 A special data structure for HTML

      4.5 Pollen command syntax

      4.6 The preprocessor

      4.7 Templated source files

      4.8 Pagetrees

    5 First tutorial

      5.1 Prerequisites

      5.2 The relationship of Racket & Pollen

      5.3 Starting a new file in DrRacket

        5.3.1 Setting the #lang line

        5.3.2 Putting in the text of the poem

        5.3.3 Saving & naming your source file

      5.4 Using the project server

        5.4.1 Starting the project server with raco pollen

        5.4.2 Using the dashboard

        5.4.3 Source files in the dashboard

      5.5 Working with the preprocessor

        5.5.1 Setting up a preprocessor source file

        5.5.2 Creating valid HTML output

        5.5.3 Adding commands

        5.5.4 Racket basics (if you’re not familiar)

        5.5.5 Defining variables with commands

        5.5.6 Inserting values from variables

        5.5.7 Inserting variables within CSS

      5.6 First tutorial complete

    6 Second tutorial

      6.1 Prerequisites

      6.2 Prelude: my principled objection to Markdown

      6.3 Markdown in Pollen: two options

        6.3.1 Using Markdown with the preprocessor

        6.3.2 Authoring mode

        6.3.3 X-expressions

        6.3.4 Markdown authoring mode

      6.4 Templates

        6.4.1 The ->html function and the doc variable

        6.4.2 Making a custom template

        6.4.3 Inserting specific source data into templates

        6.4.4 Linking to an external CSS file

      6.5 Intermission

      6.6 Pagetrees

        6.6.1 Pagetree navigation

        6.6.2 Using the automatic pagetree

        6.6.3 Adding navigation links to the template with here

        6.6.4 Handling navigation boundaries with conditionals

        6.6.5 Making a pagetree file

        6.6.6 index.ptree & the project server

      6.7 Second tutorial complete

    7 Third tutorial

      7.1 Prerequisites

      7.2 Pollen markup vs. XML

        7.2.1 The XML problem

        7.2.2 What Pollen markup does differently

        7.2.3 “But I really need XML…”

      7.3 Writing with Pollen markup

        7.3.1 Creating a Pollen markup file

        7.3.2 Tags & tag functions

        7.3.3 Attributes

        7.3.4 What are custom tags good for?

        7.3.5 Semantic markup

        7.3.6 Format independence

        7.3.7 Using custom tags

        7.3.8 Choosing custom tags

      7.4 Tags are functions

        7.4.1 Attaching behavior to tags

        7.4.2 Notes for experienced programmers

          7.4.2.1 Point of no return

          7.4.2.2 Multiple input values & rest arguments

          7.4.2.3 Returning an X-expression

          7.4.2.4 Interpolating variables into strings

          7.4.2.5 Parsing attributes

      7.5 Intermission

      7.6 Organizing functions

        7.6.1 Using Racket’s function libraries

        7.6.2 Using the directory-require.rkt file

      7.7 Decoding markup via the root tag

      7.8 Putting it all together

        7.8.1 The directory-require.rkt file

        7.8.2 The template

        7.8.3 The pagetree

        7.8.4 A CSS stylesheet using the preprocessor

        7.8.5 The content source files using Pollen markup

        7.8.6 The result

      7.9 Third tutorial complete

    8 Using raco pollen

      8.1 Making sure raco pollen works

      8.2 raco pollen

      8.3 raco pollen help

      8.4 raco pollen start

      8.5 raco pollen render

      8.6 raco pollen clone

    9 File formats

      9.1 Source formats

        9.1.1 Command syntax using ◊

        9.1.2 Any command is valid

        9.1.3 Standard exports

        9.1.4 Custom exports

        9.1.5 The directory-require.rkt file

        9.1.6 Preprocessor (.pp extension)

        9.1.7 Markdown (.pmd extension)

        9.1.8 Markup (.pm extension)

        9.1.9 Pagetree (.ptree extension)

      9.2 Utility formats

        9.2.1 Scribble (.scrbl extension)

        9.2.2 Null (.p extension)

    10 ◊ command overview

      10.1 The golden rule

      10.2 The lozenge glyph (◊)

      10.3 The two command modes: text mode & Racket mode

        10.3.1 The command name

          10.3.1.1 Invoking tag functions

          10.3.1.2 Invoking other functions

          10.3.1.3 Inserting the value of a variable

          10.3.1.4 Inserting metas

          10.3.1.5 Inserting a comment

        10.3.2 The Racket arguments

        10.3.3 The text argument

      10.4 Further reading

    11 Module reference

      11.1 Cache

      11.2 Decode

        11.2.1 Block

        11.2.2 Typography

      11.3 File

      11.4 Pagetree

        11.4.1 Making pagetrees with a source file

        11.4.2 Making pagetrees by hand

        11.4.3 Using pagetrees for navigation

        11.4.4 Using index.ptree in the dashboard

        11.4.5 Using pagetrees with raco pollen render

        11.4.6 Functions

          11.4.6.1 Predicates & validation

          11.4.6.2 Navigation

          11.4.6.3 Utilities

      11.5 Render

      11.6 Template

      11.7 Tag

      11.8 Top

      11.9 World

    12 Acknowledgments

    13 License & source code

    Index

    ← prev  up  next → 
 
\ No newline at end of file diff --git a/doc/quick-tour.html b/doc/quick-tour.html index a55d4d0..c0cc54c 100644 --- a/doc/quick-tour.html +++ b/doc/quick-tour.html @@ -2,4 +2,4 @@ 2 Quick tour
▼Pollen:   the book is a program
►2 Quick tour
On this page:
2.1 Creating a source file
2.2 Running a source file
2.3 Naming, saving, and rendering a source file
2.4 The project server
2.5 Intermission
2.6 Pollen as a preprocessor
2.7 Markdown mode
2.8 Markup mode
2.9 Templates
2.10 PS for Scribble users
2.11 The end of the beginning
6.1.0.5
    ← prev  up  next → 

2 Quick tour

2.1 Creating a source file

Assuming you’ve installed Racket & Pollen, launch DrRacket.

Open a new document. Change the top line to:

#lang pollen

The first line of every Pollen source file will start with #lang pollen.

2.2 Running a source file

Add a second line to your source file so it reads:

#lang pollen
Hello world

Click the Run button. In the interactions window, you’ll see the result:

Hello world

Not bad. I think Pollen just won the Hello World Tournament.

You can work with Pollen source files in any text editor. The key advantage of DrRacket is that you can preview the results by running the file.

Try editing your source file:

#lang pollen
Goodbye Stranger
Breakfast in America
Take the Long Way Home

You don’t have to use Supertramp song titles. Any text will do. When you click Run again, you’ll see whatever you typed:

Goodbye Stranger
Breakfast in America
Take the Long Way Home

We won’t do it a third time. You get the point — any plain text is valid within a Pollen source file, and gets printed as is. You never have to perform the incantations of typical programming languages:

print "Hello world"

document.write('Hello world');

printf("Hello world");

In Pollen, what you write is what you get.

2.3 Naming, saving, and rendering a source file

Save this file with the name hello.txt.pp in any convenient directory. The desktop is fine.

Open a terminal window and issue two commands:

> cd [directory containing your file]

> raco pollen render hello.txt.pp

After a moment, a new file will appear called hello.txt. Let’s see what’s in it:

> cat hello.txt

Goodbye Stranger

Breakfast in America

Take the Long Way Home

You’ve just learned three things:

  • Pollen commands in the terminal begin with raco pollen, followed by a specific command (in this case render) and sometimes an argument (in this case hello.txt.pp).

  • The render command takes the ouput from your source file — meaning, the result you previewed in DrRacket in the previous step — and saves it to another file.

  • The name of the output file is the same as the source file, minus the Pollen source extension. So hello.txt.pp becomes hello.txt.

Try editing the text in the hello.txt.pp source file and running raco pollen render hello.txt.pp again. The old hello.txt will be replaced with a new one showing your changes. And so you’ve learned a fourth thing:

  • Pollen works by rendering output files from source files. Output files can be overwritten. Therefore, you should only make edits to your source files.

2.4 The project server

You’ve just learned two ways to see the output of a Pollen source file — first, you ran it in DrRacket. Then, you rendered it to an output file.

Now here’s a third: the Pollen project server. Here’s how you start it. Return to your terminal window and issue two commands:

> cd [directory containing your hello.txt.pp file]

> raco pollen start

After a moment, you’ll see the startup message:

Welcome to Pollen 0.001 (Racket 6.0.0.5)

Project root is /path/to/your/directory

Project server is http://localhost:8080 (Ctrl-C to exit)

Project dashboard is http://localhost:8080/index.ptree

Ready to rock

Open a web browser and point it at http://localhost:8080/index.ptree. The top of the window will say Project root. Below that will be a listing of the files in the directory.

Among them will be hello.txt, with a greyed-out .pp extension. Click on it, and you’ll be taken to http://localhost:8080/hello.txt, where you’ll see:

Goodbye Stranger

Breakfast in America

Take the Long Way Home

That’s the boring part. Here’s the good part. Leave the project server running. Open your source file again in DrRacket and edit it as follows:

"hello.txt.pp"
#lang pollen
Mean Street
Panama
Hear About It Later

Go back to your web browser and reload http://localhost:8080/hello.txt. Now you’ll see this:

Mean Street

Panama

Hear About It Later

Notice what happened — the Pollen project server dynamically regenerated the output file (hello.txt) from the source file (hello.txt.pp) after you edited the source. If you like, try making some more changes to hello.txt.pp, and reloading the browser to see the updates in hello.txt.

2.5 Intermission

That covers input & output. Now let’s circle back and look at what else you can do with Pollen (beyond the epic achievement of displaying plain text in a web browser).

For the rest of this tutorial, I recommend keeping two windows on screen: a web-browser window pointed at your project server (the main URL is http://localhost:8080/index.ptree) and the DrRacket editing window.

2.6 Pollen as a preprocessor

A preprocessor is a tool for making systematic, automated changes to a source file before the main processing happens. A preprocessor can also be used to add programming logic to files that otherwise don’t support it.

For instance, HTML. In DrRacket, create a new file called margin.html.pp in your project directory:

"margin.html.pp"
#lang pollen
<body style="margin: 5em; border:1px solid black">
5em is the inset.
</body>

The “.pp” file extension — which you saw before, with hello.txt.pp — stands for “Pollen preprocessor.” You can use the Pollen preprocessor with any text-based file by inserting #lang pollen as the first line, and adding the .pp file extension.

But for now, go to your project dashboard and click on margin.html. You should see a black box containing the text “5em is the inset.”

Let’s suppose you want to change the inset to 30%. Without a preprocessor, you’d have to search & replace each value. But with a preprocessor, you can move the inset value into a variable, and update it from that one location. So first, introduce a variable called my-inset by using the define command:

"margin.html.pp"
#lang pollen
◊define[my-inset]{30%}
<body style="margin: 10em; border:1px solid black">
10em is the inset.
</body>

The ◊ character is called a lozenge. In Pollen, the lozenge is a special character that marks anything Pollen should interpret as a command (rather than plain text).

How to type a lozenge:
Mac: option + shift + V
Windows: holding down alt, type 9674 on the num pad -
Ubuntu: ctrl + shift + U, then 25CA

The whole command ◊define[my-inset]{30%} means “create a variable called my-inset and give it the value 30%.”

Then put the variable into the HTML like so, this time using the ◊ character with the variable name in the two places the value appears:

"margin.html.pp"
#lang pollen
◊define[my-inset]{30%}
<body style="margin: ◊my-inset; border:1px solid black">
◊my-inset is the inset.
</body>

Now reload margin.html. You’ll see that the size of the margin has changed (because of the change to the style attribute) and so has the text of the HTML. If you like, try editing my-inset with different values and reloading the page. You can also try using define to create another variable (for instance, to change the color of the box border).

Still, this is the tiniest tip of the iceberg. The Pollen preprocessor gives you access to everything in the Racket programming language — including math functions, text manipulation, and so on.

2.7 Markdown mode

When used as a preprocessor, Pollen’s rule is that what you write is what you get. But if you’re targeting HTML, who wants to type out all those <tedious>tags</tedious>? You can make Pollen do the heavy lifting by using it as a source decoder.

For instance, Markdown mode. Markdown is a simplified notation system for HTML. You can use Pollen’s Markdown decoder by inserting #lang pollen as the first line, and adding the .pmd file extension.

Try it. In DrRacket, create a file with the following lines and save it as downtown.html.pmd:

"downtown.html.pmd"
#lang pollen
 
Pollen + Markdown
-----------------
 
+ You **wanted** it — you #,(racketfont "_got_") it.
 
+ [search for Racket](https://google.com/search?q=racket)

As before, go to the dashboard for the project server. This time, click the link for downtown.html. You’ll see something like this:

Pollen + Markdown

As usual, you’re welcome to edit downtown.html.pmd and then refresh the web browser to see the changes.

In Markdown mode, you can still embed Pollen commands within the source as you did in preprocessor mode. Just keep in mind that your commands need to produce valid Markdown (as opposed to raw HTML). For instance, use define to create a variable called metal, and insert it into the Markdown:

"downtown.html.pmd"
#lang pollen
◊define[metal]{Plutonium}
 
Pollen + ◊metal
--------
 
+ You **wanted** ◊metal — you #,(racketfont "_got_") it.
 
+ [search for ◊metal](https://google.com/search?q=◊metal)

Refresh downtown.html in the browser:

Pollen + Plutonium

Pollen is handling two tasks here: interpreting the commands in the source, and then converting the Markdown to HTML. But what if you wanted to use Pollen as a preprocessor that outputs a Markdown file? No problem — just change the source name from downtown.html.pmd to downtown.md.pp. Changing the extension from .pmd to .pp switches Pollen from Markdown mode back to preprocessor mode. And changing the base name from downtown.html to downtown.md updates the name of the output file.

2.8 Markup mode

If all you need to do is produce basic HTML, Markdown is great. But if you need to do semantic markup or other kinds of custom markup, it’s not flexible enough.

In that case, you can use Pollen markup mode. To use Pollen markup, insert #lang pollen as the first line of your source file, and add a .pm file extension.

Compared to Markdown mode, Pollen markup mode is wide open. Markdown mode gives you a limited set of formatting tools (i.e., the ones supported by Markdown). But in markup mode, you can use any tags you want. Markdown mode decodes the source in a fixed way (i.e., with the Markdown decoder). But markup mode lets you build any decoder you want.

Let’s convert our Markdown example into Pollen markup. Marking up content is simple: insert the lozenge character (◊) followed by the name of the tag (◊tag), followed by the content of the tag in curly braces (◊tag{content}). In DrRacket, create a new file called uptown.html.pm as follows:

"uptown.html.pm"
#lang pollen
 
◊headline{Pollen markup}
 
◊items{
  ◊item{You ◊strong{wanted} it — you ◊em{got} it.}
  ◊item{◊link["https://google.com/search?q=racket"]{search for Racket}}
}

Go to the project dashboard and click on uptown.html. You’ll see something like this:

Pollen markup You wanted it — you got it. https://google.com/search?q=racketsearch for Racket

That’s not right. What happened?

We marked up the source using a combination of standard HTML tags (strong, em) and nonstandard ones (headline, items, item, link). This is valid Pollen markup. (In fact, if you look at the generated source, you’ll see that they didn’t disappear.) But since we’re targeting HTML, we need to convert our custom tags into valid HTML tags.

For that, we’ll make a special file called directory-require.rkt. This is a file in the standard Racket language that provides helper functions to decode the source. The definitions won’t make sense yet. But this is the quick tour, so all you need to do is copy, paste, and save:

"directory-require.rkt"
#lang racket/base
 
(require pollen/tag)
(provide (all-defined-out))
(define headline (make-default-tag-function 'h2))
(define items (make-default-tag-function 'ul))
(define item (make-default-tag-function 'li 'p))
(define (link url text) `(a [[href ,url]] ,text))

Return to the project dashboard and click on uptown.html. Now you’ll get the right result:

Pollen markup

Markup mode takes a little more effort to set up. But it also allows you more flexibility. If you want to do semantic markup, or convert your source into multiple output formats, or handle complex page layouts — it’s the way to go.

2.9 Templates

The HTML pages we just made looked pretty dull. For the last stop on the quick tour, let’s fix that.

Pollen source files that are written in Markdown or markup mode (i.e., .pmd or .pm files) are rendered with a template. A template is not a standalone Pollen source file. It’s a file of the output type — e.g., CSS, HTML, XML — where you put the stuff that needs to be consistent between output files. The template also contains template variables that mark where values from the Pollen source file should be inserted.

When it needs a template, Pollen first looks for a file in the project directory named template.[output extension of source]. For uptown.html.pm, the output extension will be .html, thus Pollen will look for template.html.

So let’s create template.html. Make a new file that with the following lines and save it to the same directory as uptown.html.pm:

"template.html"
<html><head><meta charset="UTF-8"/></head>
<body style="background: #f6f6f6">
<div style="background: white; margin: 3em;
border:10px double gray; padding: 3em; font-size: 130%;">
This file is ◊here
<hr />
◊->html{◊doc}
</div></body></html>

This is a simple HTML file that should look familiar, except for the two template variables. The first, here, contains the name of the current source file. As before, the lozenge character marks it as a Pollen command rather than text, so you write it as ◊here. The other command, ◊->html{◊doc}, takes the content from the source file, which is contained in a variable called doc, and converts it to HTML with a Pollen function called ->html.

Go back to your web browser and reload uptown.html. (Or downtown.html — both will work.) The page will be rendered with the new template.html. As before, you can edit the template or the source and the project server will dynamically update the output file.

2.10 PS for Scribble users

Pollen can also be used as a dynamic preview server for Scribble files. From your terminal, do the following:

> cd [directory containing your Scribble files]

> raco pollen start

On the project dashboard, you’ll see your [filename].scrbl files listed as [filename].html. This may not represent the ultimate structure of your Scribble project — you may end up combining multiple Scribble source files into one HTML file, or making multiple HTML files from one Scribble source — but it’s handy for checking your work as you go.

2.11 The end of the beginning

Now you’ve seen the key features of Pollen. What do you think?

  • “So it’s like WordPress, but harder to use?” I was a happy WordPress user for several years. If you need a blog, it’s great. But the farther you get from blogs, the more it becomes like teaching an elephant to pirouette. And for those who like to solve problems with programming, PHP is, um, limited.

  • “What about pairing a Python template system and Python web server?” Good idea. I even tried it. But Python template systems don’t offer you Python — they offer you pidgin dialects that ain’t very Pythonic. Also, Python’s handing of markup-based data structures is cumbersome.

  • “Haven’t you heard of Jekyll?” Yes. If everything you need to write is expressible in Markdown, it’s great. If you need more than that, you’re stuck.

  • “Sounds a lot like LaTeX. Why not use that?” Also a good idea. LaTeX gets a lot of things right. But it wasn’t designed for web publishing.

  • “Eh, there are plenty of adequate options. Why should I learn a system written in Racket, which I’ve never used?” A salient objection. It’s also the question I asked myself before I committed to Racket. But publishing systems that are author- or designer-friendly tend to be programmer-hostile, and vice versa. Racket is the only language I found that could meet my requirements.

But don’t take my word for it. The rest of this documentation will show you the cool, useful, and sophisticated things you can do with Pollen. If there’s another tool that suits you better, great. Keep in mind that I didn’t make Pollen because I’m a programmer. I’m a writer who wants to make electronic books that are better than the ones we have now. And for that, I needed a better tool.

    ← prev  up  next → 
 
\ No newline at end of file +
Ubuntu: ctrl + shift + U, then 25CA

The whole command ◊define[my-inset]{30%} means “create a variable called my-inset and give it the value 30%.”

Then put the variable into the HTML like so, this time using the ◊ character with the variable name in the two places the value appears:

"margin.html.pp"
#lang pollen
◊define[my-inset]{30%}
<body style="margin: ◊my-inset; border:1px solid black">
◊my-inset is the inset.
</body>

Now reload margin.html. You’ll see that the size of the margin has changed (because of the change to the style attribute) and so has the text of the HTML. If you like, try editing my-inset with different values and reloading the page. You can also try using define to create another variable (for instance, to change the color of the box border).

Still, this is the tiniest tip of the iceberg. The Pollen preprocessor gives you access to everything in the Racket programming language — including math functions, text manipulation, and so on.

2.7 Markdown mode

When used as a preprocessor, Pollen’s rule is that what you write is what you get. But if you’re targeting HTML, who wants to type out all those <tedious>tags</tedious>? You can make Pollen do the heavy lifting by using it as a source decoder.

For instance, Markdown mode. Markdown is a simplified notation system for HTML. You can use Pollen’s Markdown decoder by inserting #lang pollen as the first line, and adding the .pmd file extension.

Try it. In DrRacket, create a file with the following lines and save it as downtown.html.pmd:

"downtown.html.pmd"
#lang pollen
 
Pollen + Markdown
-----------------
 
+ You **wanted** it — you #,(racketfont "_got_") it.
 
+ [search for Racket](https://google.com/search?q=racket)

As before, go to the dashboard for the project server. This time, click the link for downtown.html. You’ll see something like this:

Pollen + Markdown

       • You wanted it — you got it.

       • search for Racket

As usual, you’re welcome to edit downtown.html.pmd and then refresh the web browser to see the changes.

In Markdown mode, you can still embed Pollen commands within the source as you did in preprocessor mode. Just keep in mind that your commands need to produce valid Markdown (as opposed to raw HTML). For instance, use define to create a variable called metal, and insert it into the Markdown:

"downtown.html.pmd"
#lang pollen
◊define[metal]{Plutonium}
 
Pollen + ◊metal
--------
 
+ You **wanted** ◊metal — you #,(racketfont "_got_") it.
 
+ [search for ◊metal](https://google.com/search?q=◊metal)

Refresh downtown.html in the browser:

Pollen + Plutonium

       • You wanted Plutonium — you got it.

       • search for Plutonium

Pollen is handling two tasks here: interpreting the commands in the source, and then converting the Markdown to HTML. But what if you wanted to use Pollen as a preprocessor that outputs a Markdown file? No problem — just change the source name from downtown.html.pmd to downtown.md.pp. Changing the extension from .pmd to .pp switches Pollen from Markdown mode back to preprocessor mode. And changing the base name from downtown.html to downtown.md updates the name of the output file.

2.8 Markup mode

If all you need to do is produce basic HTML, Markdown is great. But if you need to do semantic markup or other kinds of custom markup, it’s not flexible enough.

In that case, you can use Pollen markup mode. To use Pollen markup, insert #lang pollen as the first line of your source file, and add a .pm file extension.

Compared to Markdown mode, Pollen markup mode is wide open. Markdown mode gives you a limited set of formatting tools (i.e., the ones supported by Markdown). But in markup mode, you can use any tags you want. Markdown mode decodes the source in a fixed way (i.e., with the Markdown decoder). But markup mode lets you build any decoder you want.

Let’s convert our Markdown example into Pollen markup. Marking up content is simple: insert the lozenge character (◊) followed by the name of the tag (◊tag), followed by the content of the tag in curly braces (◊tag{content}). In DrRacket, create a new file called uptown.html.pm as follows:

"uptown.html.pm"
#lang pollen
 
◊headline{Pollen markup}
 
◊items{
  ◊item{You ◊strong{wanted} it — you ◊em{got} it.}
  ◊item{◊link["https://google.com/search?q=racket"]{search for Racket}}
}

Go to the project dashboard and click on uptown.html. You’ll see something like this:

Pollen markup You wanted it — you got it. https://google.com/search?q=racketsearch for Racket

That’s not right. What happened?

We marked up the source using a combination of standard HTML tags (strong, em) and nonstandard ones (headline, items, item, link). This is valid Pollen markup. (In fact, if you look at the generated source, you’ll see that they didn’t disappear.) But since we’re targeting HTML, we need to convert our custom tags into valid HTML tags.

For that, we’ll make a special file called directory-require.rkt. This is a file in the standard Racket language that provides helper functions to decode the source. The definitions won’t make sense yet. But this is the quick tour, so all you need to do is copy, paste, and save:

"directory-require.rkt"
#lang racket/base
 
(require pollen/tag)
(provide (all-defined-out))
(define headline (make-default-tag-function 'h2))
(define items (make-default-tag-function 'ul))
(define item (make-default-tag-function 'li 'p))
(define (link url text) `(a [[href ,url]] ,text))

Return to the project dashboard and click on uptown.html. Now you’ll get the right result:

Pollen markup

       • You wanted it — you got it.

       • search for Racket

Markup mode takes a little more effort to set up. But it also allows you more flexibility. If you want to do semantic markup, or convert your source into multiple output formats, or handle complex page layouts — it’s the way to go.

2.9 Templates

The HTML pages we just made looked pretty dull. For the last stop on the quick tour, let’s fix that.

Pollen source files that are written in Markdown or markup mode (i.e., .pmd or .pm files) are rendered with a template. A template is not a standalone Pollen source file. It’s a file of the output type — e.g., CSS, HTML, XML — where you put the stuff that needs to be consistent between output files. The template also contains template variables that mark where values from the Pollen source file should be inserted.

When it needs a template, Pollen first looks for a file in the project directory named template.[output extension of source]. For uptown.html.pm, the output extension will be .html, thus Pollen will look for template.html.

So let’s create template.html. Make a new file that with the following lines and save it to the same directory as uptown.html.pm:

"template.html"
<html><head><meta charset="UTF-8"/></head>
<body style="background: #f6f6f6">
<div style="background: white; margin: 3em;
border:10px double gray; padding: 3em; font-size: 130%;">
This file is ◊here
<hr />
◊->html{◊doc}
</div></body></html>

This is a simple HTML file that should look familiar, except for the two template variables. The first, here, contains the name of the current source file. As before, the lozenge character marks it as a Pollen command rather than text, so you write it as ◊here. The other command, ◊->html{◊doc}, takes the content from the source file, which is contained in a variable called doc, and converts it to HTML with a Pollen function called ->html.

Go back to your web browser and reload uptown.html. (Or downtown.html — both will work.) The page will be rendered with the new template.html. As before, you can edit the template or the source and the project server will dynamically update the output file.

2.10 PS for Scribble users

Pollen can also be used as a dynamic preview server for Scribble files. From your terminal, do the following:

> cd [directory containing your Scribble files]

> raco pollen start

On the project dashboard, you’ll see your [filename].scrbl files listed as [filename].html. This may not represent the ultimate structure of your Scribble project — you may end up combining multiple Scribble source files into one HTML file, or making multiple HTML files from one Scribble source — but it’s handy for checking your work as you go.

2.11 The end of the beginning

Now you’ve seen the key features of Pollen. What do you think?

But don’t take my word for it. The rest of this documentation will show you the cool, useful, and sophisticated things you can do with Pollen. If there’s another tool that suits you better, great. Keep in mind that I didn’t make Pollen because I’m a programmer. I’m a writer who wants to make electronic books that are better than the ones we have now. And for that, I needed a better tool.

    ← prev  up  next → 
 
\ No newline at end of file diff --git a/doc/second-tutorial.html b/doc/second-tutorial.html index 577f1b2..10f4151 100644 --- a/doc/second-tutorial.html +++ b/doc/second-tutorial.html @@ -1,5 +1,5 @@ -6 Second tutorial
▼Pollen:   the book is a program
►6 Second tutorial
On this page:
6.1 Prerequisites
6.2 Prelude:   my principled objection to Markdown
6.3 Markdown in Pollen:   two options
6.3.1 Using Markdown with the preprocessor
6.3.2 Authoring mode
6.3.3 X-expressions
6.3.4 Markdown authoring mode
6.4 Templates
6.4.1 The ->html function and the doc variable
6.4.2 Making a custom template
6.4.3 Inserting specific source data into templates
6.4.4 Linking to an external CSS file
6.5 Intermission
6.6 Pagetrees
6.6.1 Pagetree navigation
6.6.2 Using the automatic pagetree
6.6.3 Adding navigation links to the template with here
6.6.4 Handling navigation boundaries with conditionals
6.6.5 Making a pagetree file
6.6.6 index.ptree & the project server
6.7 Second tutorial complete
6.1.0.5
    ← prev  up  next → 

6 Second tutorial

In this tutorial, you’ll use Pollen to publish a multiple-page article written in Markdown. You’ll learn about:

  • Using Markdown files with the preprocessor

  • X-expressions

  • Markdown authoring mode

  • Templates

  • Pagetrees

If you want the shortest possible introduction to Pollen, try the Quick tour.

6.1 Prerequisites

I’ll assume you’ve completed the First tutorial and you understand how to create source files in DrRacket and view them in the project server. I will not be spelling out those tasks as I did before.

6.2 Prelude: my principled objection to Markdown

I recognize that people like Markdown. I want people to like Pollen too, so that’s why Markdown support exists in Pollen. But just to be clear about my own views —

I’m mystified by the popularity of Markdown among writers. I can agree that it’s a clever and readable way of notating basic HTML. And sure, that makes it great for things like web comments, where speed and simplicity are primary virtues.

In longer-form writing, however, its shortcomings become evident. Like programming languages, the best writing tools maximize expressive possibilities, and minimize constraints. But Markdown is hugely constrained. First and worst, Markdown isn’t semantic. It only knows about formatting, and in that regard, isn’t that much of an improvement on tools like Microsoft Word. Second, even as a formatting-notation tool, it’s limited to a small subset of the already-small set of formatting tags permitted in HTML. Third, it can’t be extended by an author.

An animating principle of Pollen, as explained in the Backstory, is that after 20 years, we ought to move beyond thinking of HTML as a source format. Since Markdown is just well-disguised HTML, a vote for Markdown is really a vote to continue the status quo (albeit with fewer angle brackets). For me, that’s not good enough. I’m ready for the tools to expand to fit my ideas; I don’t want to keep cutting down my ideas to fit the tools.

All that said, if you genuinely prefer Markdown, I’m not looking to pry it from your fingers. Pollen has excellent Markdown support (due entirely to Greg Hendershott’s excellent Markdown parser for Racket). It makes Markdown more useful.

But let’s make a deal, Markdown fans. Having met you more than halfway, will you at least consider that Pollen markup might be a better option for you than Markdown? Because it can notate anything that’s in your brain, not just a subset of HTML? And if The book is a program, the source for that book should look more like your brain, and less like HTML?

That’s all I ask.

6.3 Markdown in Pollen: two options

There are two ways to use Markdown within Pollen: you can either send Markdown files through the preprocessor, or use Markdown authoring mode.

The preprocessor approach is better if you want to end up with a set of Markdown files that can be passed along to a HTML converter (or other Markdown-to-______ converter) elsewhere.

The authoring-mode approach is better if you want to end up with something other than Markdown, e.g., finished HTML files.

6.3.1 Using Markdown with the preprocessor

Because Markdown is a text-based format, you can use the Pollen preprocessor to add programmatic features to existing Markdown files. (See Working with the preprocessor in the First tutorial if you need a refresher.)

Suppose we have a Markdown file called brennan.md that we want to use with the preprocessor. Create this file in DrRacket, save it, and start the project server in that directory.

"brennan.md"
My name is _Brennan_, and I enjoy:
 
+ boring sauce
 
+ 24 fish nuggets

You’ll be able to see this file in the project server, but for now, it’s just a static file. Pollen isn’t doing anything to it.

Let’s change that. Consistent with the usual preprocessor practice, we add #lang pollen as the first line, and append the .pp file extension, so our new preprocessor-ready file looks like this:

"brennan.md.pp"
#lang pollen
 
My name is _Brennan_, and I enjoy:
 
+ boring sauce
 
+ 24 fish nuggets

Go back to the project server and you’ll see the new filename. When you click on it, Pollen will render a new markdown.md file, but it will look the same as the one you had before.

Now we’ll change some of the values using Pollen commands:

"brennan.md.pp"
#lang pollen
 
◊(define sauce-type "fancy")
◊(define nugget-type "chicken")
◊(define nugget-quantity (* 2 2 3))
 
My name is _Brennan_, and I enjoy:
 
+ ◊sauce-type sauce
 
+ ◊nugget-quantity ◊nugget-type nuggets

When you reload this file in the project server, brennan.md will be regenerated, and will now look like this:

My name is _Brennan_, and I enjoy:

 

+ fancy sauce

 

+ 12 chicken nuggets

Instead of running Markdown files through the preprocessor, you can also use Markdown authoring mode within Pollen. This is the better choice if you want to end up with rendered HTML files.

But first, let’s pause to clarify the general concept of an authoring mode.

6.3.2 Authoring mode

Though the preprocessor is useful, it limits you to inserting chunks of text at various positions into an existing file.

Pollen’s authoring mode, by contrast, parses the whole source file into a special data structure called an X-expression. You can then process the whole X-expression any way you like, and output to any format you like — or multiple formats — using a template.

Compared to the preprocessor, authoring mode offers more abstraction and flexibility. Of course, it’s also a bit more effort to set up.

Pollen offers two variants of authoring mode: one that uses Markdown syntax (which we’ll cover later in this tutorial) and the other that uses a free-form markup syntax (which we’ll cover in the next tutorial). In both cases, the basic idea is the same: parse the source into an X-expression, and then output it using a template.

6.3.3 X-expressions

Don’t skip this section! It explains a concept that’s essential to understanding how Pollen works.

I avoid nerdy jargon whenever possible. But in this case, the thing is called an X-expression throughout the Racket documentation, for good reasons. So I use the term too. Better to acclimate you now.

An X-expression is a way of representing markup-based data in code. X-expressions are indigenous to Lisp-based languages like Pollen and Racket. They don’t exist in Python or JavaScript or Ruby.

Let’s start with the part you’re familiar with. By “markup-based data,” I mean things like HTML and XML and SVG. The idea is that you have text-based data surrounded by tags. Each tag can also have its own attributes that are made of keys and values. Tags can contain other tags, thus creating a tree-like structure. Right? You know what I mean:

<body><h1>Hello world</h1><p class="first">Nice to <i>see</i> you.</p></body>

An X-expression is just a simplified, generalized method of notation for these data structures — much like Markdown is a simplified method of notation for HTML. To see the relationship, we’ll convert one into the other.

First, we change the angle brackets to parentheses, and only use them on the outside of tags:

(body (h1 Hello world /h1) (p class="first" Nice to (i see /i) you. /p) /body)

Then we get rid of the closing tags, which are superfluous, since each closing parenthesis adequately marks the end of the tag:

(body (h1 Hello world) (p class="first" Nice to (i see) you.))

However, this creates ambiguity between the name of the tag and the content. So we’ll put the content within double quotes:

(body (h1 "Hello world") (p class="first" "Nice to" (i "see") "you."))

As for the class attribute, we need to distinguish it from both the markup tags and the content, so we’ll move it between double parentheses:

(body (h1 "Hello world") (p ((class "first")) "Nice to" (i "see") "you."))

Net of a few boring details, that’s basically all there is to it.

So why is it called an X-expression? Lisp languages are built out of units called S-expressions, which look like this:

(and (txexpr? x) (member (get-tag x) (project-block-tags)) #t))

S-expressions use prefix notation, where each pair of parentheses contains a list. The first element in the list names a function, and the other elements are the arguments to that function. (This is a review of Racket basics (if you’re not familiar).) X-expressions are just a minor adaptation of S-expression notation to represent markup, hence the name (the X is short for XML-like).

For handling markup-based data, X-expressions have some useful advantages compared to other methods:

  • Readability. X-expressions retain all the semantics of markup-based data while dispensing with the infamous verbosity.

  • A hybrid between a tree and a string. Most programming languages represent markup-based data either as a string or as an XML tree. Neither is a good choice. The string captures none of the internal structure of the data. An XML tree captures the structure, but conceals the sequential nature of the data elements. The X-expression gets both.

  • An ideal match for an expression-based programming language. Aside from some notational details, X-expressions are just a subset of S-expressions generally, which are the building block of Racket. Processing X-expressions in Racket maximizes flexibility and minimizes yak-shaving.

Given the close kinship between XML-ish data structures and Lisp-ish programming languages, I have no explanation why, during the Internet era, they have not been paired more often.

In Pollen’s authoring modes, your source file is parsed into an X-expression, which can then be processed further before being injected into a template & converted to output. As a first example, we’ll look at Markdown authoring mode.

6.3.4 Markdown authoring mode

Let’s start putting together our article. For simplicity, I’m going to use unrealistically short sample texts. But you can use whatever Markdown content you want.

We want to use Markdown authoring mode to make a file that will ultimately be HTML. So consistent with Pollen file-naming conventions (see Saving & naming your source file), we’ll start with our desired output filename, article.html, and then append the Markdown authoring suffix, .pmd. So in DrRacket, start a new file called article.html.pmd and put some Markdown in it:

"article.html.pmd"
#lang pollen
 
Deep Thought
============
 
I am **so** happy to be writing this.

Before you preview this file in the project server, click the Run button in DrRacket just to see what the file produces. You’ll see something like this:

'(root (h1 ((id "my-article")) "Deep Thought") (p () "I am "
(strong () "so") " happy to be writing this."))

You should now be able to recognize this as an X-expression. In authoring mode, Pollen parses your Markdown into the corresponding HTML entities, but then provides the data as an X-expression rather than finished HTML.

The empty parentheses () after p and strong signal that the tag’s attributes are empty. When you write an X-expression without attributes, these parentheses are optional — (tag () "text") and (tag "text") are equivalent — but Pollen will always print X-expressions this way.

From what you learned in the last section, it should be evident that this X-expression corresponds to HTML that looks like this:

<root><h1 id="my-article">Deep Thought</h1><p>I am
<strong>so</strong> happy to be writing this.</p></root>

“But what’s this root tag? That’s not HTML.” An X-expression must have a root tag, so in the spirit of obviousness, every X-expression produced by a source file in authoring mode will start with root. If you don’t need it, you can discard it. But it also creates a useful hook for further processing, as we’ll see later.

By the way, as review, let’s remind ourselves how this is different from preprocessor mode. Let’s take the same Markdown content, but this time put it into a preprocessor source file called article.md.pp.

"article.md.pp"
#lang pollen
 
Deep Thought
============
 
I am **so** happy to be writing this.

When you run this file in DrRacket, you’ll see:

Deep Thought
============

I am **so** happy to be writing this.

Hopefully, this result makes sense to you: when you run Markdown source in preprocessor mode, you get Markdown. When you run Markdown source in authoring mode, you get an X-expression.

6.4 Templates

So how do you convert an X-expression into a finished file? You use a Pollen template, which takes data from an X-expression and converts it to the target format.

If you’ve used other web-publishing systems, templates are probably a familiar idea. Templates in Pollen are in some ways similar to the ones you’ve seen before, but in other ways different.

First, the similarities. At its simplest, a template holds boilerplate material that you want to reuse across multiple pages. For instance, in a set of HTML pages, you might have layout and navigation elements that stay the same, while the content changes. In that case, you could put the layout and navigation in the template, and keep the content in your Pollen source files. When you want to add a new page, you can make a new source file and just use it with the existing template. Moreover, if you want to change the layout and navigation globally, you can just change the template, rather than changing the source files.

Pollen templates, like others, can also have conditional features — meaning, you can embed simple code in your templates that allows them to change based on the content in the page. For instance, a template could show or hide “previous page” and “next page” links depending on whether there’s actually a previous or next page.

The major difference with Pollen templates is that there’s no special “template language” you need to learn, with magic delimiters and whatnot. Instead, you can use all the same Pollen commands in a template that you can in authoring mode or preprocessor mode.

To see a template in action, let’s return to the source file we started in the last section:

"article.html.pmd"
#lang pollen
 
Deep Thought
============
 
I am **so** happy to be writing this.

Last time, I had you run this file in DrRacket to see the X-expression it produced. This time, load it in the project server. You’ll see something like this:

Deep Thought

 

I am so happy to be writing this.

Here, you’re seeing the X-expression from your source combined with an HTML template, which adds the necessary boilerplate for the finished HTML:

<html><head><meta charset="UTF-8" /></head><body>
<root><h1 id="my-article">Deep Thought</h1><p>I am
<strong>so</strong> happy to be writing this.</p></root>
</body></html>

But wait — where did the template come from? When you view an authoring-mode source file in the project server without specifying a template, Pollen helps you out and uses its fallback template. The fallback template is just a minimal template that’s used as a last resort. Under ordinary circumstances, seeing the fallback template usually signals a problem (e.g., Pollen couldn’t find the template you asked for).

But we can learn a few things from the fallback template about how to make an HTML template.

6.4.1 The ->html function and the doc variable

This is the fallback template that Pollen uses.

"fallback.html"
◊(->html (html (head (meta 'charset: "UTF-8")) (body doc)))

It has three key ingredients.

First, there’s an X-expression that represents a basic HTML page:

(html (head (meta 'charset: "UTF-8")) (body))

This is equivalent to the HTML:

<html><head><meta charset="UTF-8"></head><body></body></html>

But within a template, we need to explicitly convert from X-expression to HTML. So we wrap this X-expression with our second key ingredient, the Pollen command ->html:

◊(->html (html (head (meta 'charset: "UTF-8")) (body)))

Third, we need to include the content from our source file. We do this by putting the variable doc inside the body tag.

◊(->html (html (head (meta 'charset: "UTF-8")) (body doc)))

By convention, every Pollen source file makes its output available through the variable doc. A source file in preprocessor mode puts its text result in doc. And a source file in authoring mode puts its X-expression result in doc.

You can change the name to something other than doc by changing world:main-pollen-export.

Under the hood, a template is just a partial program that relies on a set of variables defined by another source file (fancy name: lexical context). So if you ran this template on its own, nothing would happen, because doc isn’t defined. But when you run it in the context of another source file, it picks up the doc that’s defined by that file.

Caution — despite the name, a Pollen template is not necessarily a file of the type suggested by its extension. For instance, fallback.html is a file that ultimately produces HTML, but it’s not actually written in HTML.

It could be, however. Here’s an equivalent way of writing fallback.html that inserts doc into actual HTML, rather than making the whole thing an X-expression.

<html><head><meta charset="UTF-8"></head>
<body>◊(->html doc)</body></html>

Notice that we still need to use the ->html function, but this time, instead of surrounding a larger X-expression, it just goes around doc.

Truly, there is no difference between these two methods. Use whichever works best for you. I often prefer the second method because I like to build & test HTML layouts by hand using placeholder content to make sure all the fiddly bits work. Then it’s easy to replace the placeholder content with (->html doc), and it becomes a template.

6.4.2 Making a custom template

We’ll use these three ingredients to make our own template for article.html.pmd.

In general, template files can have any name you want. But by default, Pollen will first look for a file in your project directory called template.ext, where ext matches the output-file extension of the source file. So if your source file is database.xml.pmd, Pollen will look for template.xml. And for article.html.pmd, Pollen will look for template.html.

Therefore, to set up a custom template, all we need to do is create a file called template.html in our project directory, and make sure it has the three key ingredients we saw in the fallback template. Pollen will automatically apply it to article.html.pmd when we view it in the project server.

But don’t take my word for it. In your project directory, create a new file called template.html:

"template.html"
<html>
<head><meta charset="UTF-8">
<title>Custom template</title></head>
<body>◊(->html doc)</body>
</html>

Recall from the last section that this is the same as the fallback template, but written out in HTML, and with a title element added. In fact, you can now refresh article.html in the project server. Does it look different? No — it won’t, because the resulting template is the same. You should notice, however, that the title of the browser window is now “Custom template,” because Pollen is relying on your new template file, rather than the fallback template.

Let’s change our custom template by adding a style block:

"template.html"
<html>
<head><meta charset="UTF-8">
<title>Custom template</title>
<style type="text/css">
body {padding: 3em; font-size: 20px;}
h1 {background: gray; color: white;}
strong {color: red;}
</style></head>
<body>◊(->html doc)</body>
</html>

When you refresh article.html in the project server, you’ll see that the heading now has a gray background, and one word in the text is red.

Feel free to add other settings to template.html, or update the text in article.html, and see how the page changes. As you’d expect, the project server keeps an eye on both your source files and your template files, and if one changes, it will refresh the output file automatically.

6.4.3 Inserting specific source data into templates

In the last example, we used doc to insert the entire content of the source file — as an X-expression — into the template.

But what if you want to only insert part of your source file into the template? For instance, you’ll look like a dork if the title on each page is “Custom template.” So let’s fix that.

When you’re working in a template, Pollen provides a select function that lets you extract the content of a specific tag, like so: ◊(select ’tag-name doc), which means “get the content of tag-name out of doc and put it here.”

Let’s suppose that we’d rather use the name of the article — Deep Thought — as the page title. We’re going to put a ◊(select ...) command inside the <title> tag.

Beyond that, we just need to know the tag name that contains the title. If we have a little Markdown expertise, we might already know that this part of our Markdown source:

Deep Thought
============

is going to produce a tag named h1.

What if we don’t have all the Markdown conversions memorized? No problem. We can still figure out the tag name by running the article.html.pmd source file in DrRacket and looking at the X-expression that results:

'(root (h1 ((id "my-article")) "Deep Thought") (p () "I am "
(strong () "so") " happy to be writing this."))

Either way, now we know that the text Deep Thought lives in the h1 tag. So we update our template accordingly (for brevity, I’m going to omit the style tag in these examples, but it’s fine to leave it in):

"template.html"
<html>
<head><meta charset="UTF-8">
<title>◊(select 'h1 doc)</title></head>
<body>◊(->html doc)</body>
</html>

When you refresh the page in the project server, the page title will now appear as “Deep Thought.” Of course, you can also combine static and dynamic elements in your template, like so:

"template.html"
<html>
<head><meta charset="UTF-8">
<title>◊(select 'h1 doc), by MB</title></head>
<body>◊(->html doc)</body>
</html>

The page title will now be “Deep Thought, by MB”.

A couple notes on command syntax. We inserted the select and ->html commands using Racket-mode syntax. We could also use text-mode syntax and write the commands this way:

"template.html"
<html>
<head><meta charset="UTF-8">
<title>◊select['h1 doc], by MB</title></head>
<body>◊->html[doc]</body>
</html>

This is exactly equivalent to the previous example. Skeptics are welcome to confirm this by checking the result in the project server.

Finally, notice that in the select command, the tag name 'h1 is written with a quote mark, whereas doc is not. This is an easy place to get tripped up, but the rule is simple: you don’t use a quote mark when you’re referring to the name of an existing function or variable (like select or doc). But you do need a quote mark when you’re using the text as a literal value.

Racket (and hence Pollen) makes a distinction between Symbols (e.g. 'h1) and Strings (e.g. "h1"). Without getting into the weeds, just note for now that the tag of an X-expression is always a symbol, not a string. But if you write ◊(select "h1" doc), the command will still work, because Pollen will treat it as ◊(select 'h1 doc), consistent with a general policy of not being persnickety about input types when the intention is clear.

6.4.4 Linking to an external CSS file

If you’re a super web hotshot, you probably don’t put your CSS selectors in the <head> tag. Instead, you link to an external CSS file. So it will not surprise you that in Pollen, you can do this by adding the usual <link> tag to your HTML template, in this case a file called styles.css:

"template.html"
<html>
<head><meta charset="UTF-8">
<title>◊select['h1 doc], by MB</title></head>
<link rel="stylesheet" type="text/css" media="all" href="styles.css" />
<body>◊->html[doc]</body>
</html>

Fans of hand-coded CSS, I trust you to take it from here: make your styles.css file, and enjoy the results.

But folks who paid attention during the First tutorial might be wondering “Can we link to a dynamically generated styles.css.pp file?”

Yes, of course. Here’s the rule of thumb: when you’re making links between files — whether CSS, or HTML, or anything else — Pollen doesn’t care whether the file is static or dynamic. You just refer to it by its ultimate name, in this case styles.css. If a static styles.css file exists, Pollen will use that. If it doesn’t, Pollen will look for a source file it can use to make styles.css, and generate it on the spot. (You can also start with a static file, and change it to be dynamic later, and Pollen will do the right thing.)

So to use a dynamic CSS file, we don’t need to make any changes to template.html. We just need to add styles.css.pp to the project directory:

"styles.css.pp"
#lang pollen
 
◊(define h1-color "blue")
◊(define strong-color "green")
 
body {padding: 3em; font-size: 20px;}
h1 {background: ◊|h1-color|; color: white;}
strong {color: ◊|strong-color|;}

Now, when you refresh article.html in the project server, Pollen will generate the styles.css file it needs, and you’ll see the new colors in the page. As before, if you update styles.css.pp, Pollen will notice and regenerate the CSS file when you refresh the page.

Can you add multiple dynamic style sheets? Yes. +6 Second tutorial

▼Pollen:   the book is a program
►6 Second tutorial
On this page:
6.1 Prerequisites
6.2 Prelude:   my principled objection to Markdown
6.3 Markdown in Pollen:   two options
6.3.1 Using Markdown with the preprocessor
6.3.2 Authoring mode
6.3.3 X-expressions
6.3.4 Markdown authoring mode
6.4 Templates
6.4.1 The ->html function and the doc variable
6.4.2 Making a custom template
6.4.3 Inserting specific source data into templates
6.4.4 Linking to an external CSS file
6.5 Intermission
6.6 Pagetrees
6.6.1 Pagetree navigation
6.6.2 Using the automatic pagetree
6.6.3 Adding navigation links to the template with here
6.6.4 Handling navigation boundaries with conditionals
6.6.5 Making a pagetree file
6.6.6 index.ptree & the project server
6.7 Second tutorial complete
6.1.0.5
    ← prev  up  next → 

6 Second tutorial

In this tutorial, you’ll use Pollen to publish a multiple-page article written in Markdown. You’ll learn about:

  • Using Markdown files with the preprocessor

  • X-expressions

  • Markdown authoring mode

  • Templates

  • Pagetrees

If you want the shortest possible introduction to Pollen, try the Quick tour.

6.1 Prerequisites

I’ll assume you’ve completed the First tutorial and you understand how to create source files in DrRacket and view them in the project server. I will not be spelling out those tasks as I did before.

6.2 Prelude: my principled objection to Markdown

I recognize that people like Markdown. I want people to like Pollen too, so that’s why Markdown support exists in Pollen. But just to be clear about my own views —

I’m mystified by the popularity of Markdown among writers. I can agree that it’s a clever and readable way of notating basic HTML. And sure, that makes it great for things like web comments, where speed and simplicity are primary virtues.

In longer-form writing, however, its shortcomings become evident. Like programming languages, the best writing tools maximize expressive possibilities, and minimize constraints. But Markdown is hugely constrained. First and worst, Markdown isn’t semantic. It only knows about formatting, and in that regard, isn’t that much of an improvement on tools like Microsoft Word. Second, even as a formatting-notation tool, it’s limited to a small subset of the already-small set of formatting tags permitted in HTML. Third, it can’t be extended by an author.

An animating principle of Pollen, as explained in the Backstory, is that after 20 years, we ought to move beyond thinking of HTML as a source format. Since Markdown is just well-disguised HTML, a vote for Markdown is really a vote to continue the status quo (albeit with fewer angle brackets). For me, that’s not good enough. I’m ready for the tools to expand to fit my ideas; I don’t want to keep cutting down my ideas to fit the tools.

All that said, if you genuinely prefer Markdown, I’m not looking to pry it from your fingers. Pollen has excellent Markdown support (due entirely to Greg Hendershott’s excellent Markdown parser for Racket). It makes Markdown more useful.

But let’s make a deal, Markdown fans. Having met you more than halfway, will you at least consider that Pollen markup might be a better option for you than Markdown? Because it can notate anything that’s in your brain, not just a subset of HTML? And if The book is a program, the source for that book should look more like your brain, and less like HTML?

That’s all I ask.

6.3 Markdown in Pollen: two options

There are two ways to use Markdown within Pollen: you can either send Markdown files through the preprocessor, or use Markdown authoring mode.

The preprocessor approach is better if you want to end up with a set of Markdown files that can be passed along to a HTML converter (or other Markdown-to-______ converter) elsewhere.

The authoring-mode approach is better if you want to end up with something other than Markdown, e.g., finished HTML files.

6.3.1 Using Markdown with the preprocessor

Because Markdown is a text-based format, you can use the Pollen preprocessor to add programmatic features to existing Markdown files. (See Working with the preprocessor in the First tutorial if you need a refresher.)

Suppose we have a Markdown file called brennan.md that we want to use with the preprocessor. Create this file in DrRacket, save it, and start the project server in that directory.

"brennan.md"
My name is _Brennan_, and I enjoy:
 
+ boring sauce
 
+ 24 fish nuggets

You’ll be able to see this file in the project server, but for now, it’s just a static file. Pollen isn’t doing anything to it.

Let’s change that. Consistent with the usual preprocessor practice, we add #lang pollen as the first line, and append the .pp file extension, so our new preprocessor-ready file looks like this:

"brennan.md.pp"
#lang pollen
 
My name is _Brennan_, and I enjoy:
 
+ boring sauce
 
+ 24 fish nuggets

Go back to the project server and you’ll see the new filename. When you click on it, Pollen will render a new markdown.md file, but it will look the same as the one you had before.

Now we’ll change some of the values using Pollen commands:

"brennan.md.pp"
#lang pollen
 
◊(define sauce-type "fancy")
◊(define nugget-type "chicken")
◊(define nugget-quantity (* 2 2 3))
 
My name is _Brennan_, and I enjoy:
 
+ ◊sauce-type sauce
 
+ ◊nugget-quantity ◊nugget-type nuggets

When you reload this file in the project server, brennan.md will be regenerated, and will now look like this:

My name is _Brennan_, and I enjoy:

 

+ fancy sauce

 

+ 12 chicken nuggets

Instead of running Markdown files through the preprocessor, you can also use Markdown authoring mode within Pollen. This is the better choice if you want to end up with rendered HTML files.

But first, let’s pause to clarify the general concept of an authoring mode.

6.3.2 Authoring mode

Though the preprocessor is useful, it limits you to inserting chunks of text at various positions into an existing file.

Pollen’s authoring mode, by contrast, parses the whole source file into a special data structure called an X-expression. You can then process the whole X-expression any way you like, and output to any format you like — or multiple formats — using a template.

Compared to the preprocessor, authoring mode offers more abstraction and flexibility. Of course, it’s also a bit more effort to set up.

Pollen offers two variants of authoring mode: one that uses Markdown syntax (which we’ll cover later in this tutorial) and the other that uses a free-form markup syntax (which we’ll cover in the next tutorial). In both cases, the basic idea is the same: parse the source into an X-expression, and then output it using a template.

6.3.3 X-expressions

Don’t skip this section! It explains a concept that’s essential to understanding how Pollen works.

I avoid nerdy jargon whenever possible. But in this case, the thing is called an X-expression throughout the Racket documentation, for good reasons. So I use the term too. Better to acclimate you now.

An X-expression is a way of representing markup-based data in code. X-expressions are indigenous to Lisp-based languages like Pollen and Racket. They don’t exist in Python or JavaScript or Ruby.

Let’s start with the part you’re familiar with. By “markup-based data,” I mean things like HTML and XML and SVG. The idea is that you have text-based data surrounded by tags. Each tag can also have its own attributes that are made of keys and values. Tags can contain other tags, thus creating a tree-like structure. Right? You know what I mean:

<body><h1>Hello world</h1><p class="first">Nice to <i>see</i> you.</p></body>

An X-expression is just a simplified, generalized method of notation for these data structures — much like Markdown is a simplified method of notation for HTML. To see the relationship, we’ll convert one into the other.

First, we change the angle brackets to parentheses, and only use them on the outside of tags:

(body (h1 Hello world /h1) (p class="first" Nice to (i see /i) you. /p) /body)

Then we get rid of the closing tags, which are superfluous, since each closing parenthesis adequately marks the end of the tag:

(body (h1 Hello world) (p class="first" Nice to (i see) you.))

However, this creates ambiguity between the name of the tag and the content. So we’ll put the content within double quotes:

(body (h1 "Hello world") (p class="first" "Nice to" (i "see") "you."))

As for the class attribute, we need to distinguish it from both the markup tags and the content, so we’ll move it between double parentheses:

(body (h1 "Hello world") (p ((class "first")) "Nice to" (i "see") "you."))

Net of a few boring details, that’s basically all there is to it.

So why is it called an X-expression? Lisp languages are built out of units called S-expressions, which look like this:

(and (txexpr? x) (member (get-tag x) (project-block-tags)) #t))

S-expressions use prefix notation, where each pair of parentheses contains a list. The first element in the list names a function, and the other elements are the arguments to that function. (This is a review of Racket basics (if you’re not familiar).) X-expressions are just a minor adaptation of S-expression notation to represent markup, hence the name (the X is short for XML-like).

For handling markup-based data, X-expressions have some useful advantages compared to other methods:

  • Readability. X-expressions retain all the semantics of markup-based data while dispensing with the infamous verbosity.

  • A hybrid between a tree and a string. Most programming languages represent markup-based data either as a string or as an XML tree. Neither is a good choice. The string captures none of the internal structure of the data. An XML tree captures the structure, but conceals the sequential nature of the data elements. The X-expression gets both.

  • An ideal match for an expression-based programming language. Aside from some notational details, X-expressions are just a subset of S-expressions generally, which are the building block of Racket. Processing X-expressions in Racket maximizes flexibility and minimizes yak-shaving.

Given the close kinship between XML-ish data structures and Lisp-ish programming languages, I have no explanation why, during the Internet era, they have not been paired more often.

In Pollen’s authoring modes, your source file is parsed into an X-expression, which can then be processed further before being injected into a template & converted to output. As a first example, we’ll look at Markdown authoring mode.

6.3.4 Markdown authoring mode

Let’s start putting together our article. For simplicity, I’m going to use unrealistically short sample texts. But you can use whatever Markdown content you want.

We want to use Markdown authoring mode to make a file that will ultimately be HTML. So consistent with Pollen file-naming conventions (see Saving & naming your source file), we’ll start with our desired output filename, article.html, and then append the Markdown authoring suffix, .pmd. So in DrRacket, start a new file called article.html.pmd and put some Markdown in it:

"article.html.pmd"
#lang pollen
 
Deep Thought
============
 
I am **so** happy to be writing this.

Before you preview this file in the project server, click the Run button in DrRacket just to see what the file produces. You’ll see something like this:

'(root (h1 ((id "my-article")) "Deep Thought") (p () "I am "
(strong () "so") " happy to be writing this."))

You should now be able to recognize this as an X-expression. In authoring mode, Pollen parses your Markdown into the corresponding HTML entities, but then provides the data as an X-expression rather than finished HTML.

The empty parentheses () after p and strong signal that the tag’s attributes are empty. When you write an X-expression without attributes, these parentheses are optional — (tag () "text") and (tag "text") are equivalent — but Pollen will always print X-expressions this way.

From what you learned in the last section, it should be evident that this X-expression corresponds to HTML that looks like this:

<root><h1 id="my-article">Deep Thought</h1><p>I am
<strong>so</strong> happy to be writing this.</p></root>

“But what’s this root tag? That’s not HTML.” An X-expression must have a root tag, so in the spirit of obviousness, every X-expression produced by a source file in authoring mode will start with root. If you don’t need it, you can discard it. But it also creates a useful hook for further processing, as we’ll see later.

By the way, as review, let’s remind ourselves how this is different from preprocessor mode. Let’s take the same Markdown content, but this time put it into a preprocessor source file called article.md.pp.

"article.md.pp"
#lang pollen
 
Deep Thought
============
 
I am **so** happy to be writing this.

When you run this file in DrRacket, you’ll see:

Deep Thought
============

I am **so** happy to be writing this.

Hopefully, this result makes sense to you: when you run Markdown source in preprocessor mode, you get Markdown. When you run Markdown source in authoring mode, you get an X-expression.

6.4 Templates

So how do you convert an X-expression into a finished file? You use a Pollen template, which takes data from an X-expression and converts it to the target format.

If you’ve used other web-publishing systems, templates are probably a familiar idea. Templates in Pollen are in some ways similar to the ones you’ve seen before, but in other ways different.

First, the similarities. At its simplest, a template holds boilerplate material that you want to reuse across multiple pages. For instance, in a set of HTML pages, you might have layout and navigation elements that stay the same, while the content changes. In that case, you could put the layout and navigation in the template, and keep the content in your Pollen source files. When you want to add a new page, you can make a new source file and just use it with the existing template. Moreover, if you want to change the layout and navigation globally, you can just change the template, rather than changing the source files.

Pollen templates, like others, can also have conditional features — meaning, you can embed simple code in your templates that allows them to change based on the content in the page. For instance, a template could show or hide “previous page” and “next page” links depending on whether there’s actually a previous or next page.

The major difference with Pollen templates is that there’s no special “template language” you need to learn, with magic delimiters and whatnot. Instead, you can use all the same Pollen commands in a template that you can in authoring mode or preprocessor mode.

To see a template in action, let’s return to the source file we started in the last section:

"article.html.pmd"
#lang pollen
 
Deep Thought
============
 
I am **so** happy to be writing this.

Last time, I had you run this file in DrRacket to see the X-expression it produced. This time, load it in the project server. You’ll see something like this:

Deep Thought

I am so happy to be writing this.

Here, you’re seeing the X-expression from your source combined with an HTML template, which adds the necessary boilerplate for the finished HTML:

<html><head><meta charset="UTF-8" /></head><body>
<root><h1 id="my-article">Deep Thought</h1><p>I am
<strong>so</strong> happy to be writing this.</p></root>
</body></html>

But wait — where did the template come from? When you view an authoring-mode source file in the project server without specifying a template, Pollen helps you out and uses its fallback template. The fallback template is just a minimal template that’s used as a last resort. Under ordinary circumstances, seeing the fallback template usually signals a problem (e.g., Pollen couldn’t find the template you asked for).

But we can learn a few things from the fallback template about how to make an HTML template.

6.4.1 The ->html function and the doc variable

This is the fallback template that Pollen uses.

"fallback.html"
◊(->html (html (head (meta 'charset: "UTF-8")) (body doc)))

It has three key ingredients.

First, there’s an X-expression that represents a basic HTML page:

(html (head (meta 'charset: "UTF-8")) (body))

This is equivalent to the HTML:

<html><head><meta charset="UTF-8"></head><body></body></html>

But within a template, we need to explicitly convert from X-expression to HTML. So we wrap this X-expression with our second key ingredient, the Pollen command ->html:

◊(->html (html (head (meta 'charset: "UTF-8")) (body)))

Third, we need to include the content from our source file. We do this by putting the variable doc inside the body tag.

◊(->html (html (head (meta 'charset: "UTF-8")) (body doc)))

By convention, every Pollen source file makes its output available through the variable doc. A source file in preprocessor mode puts its text result in doc. And a source file in authoring mode puts its X-expression result in doc.

You can change the name to something other than doc by changing world:main-pollen-export.

Under the hood, a template is just a partial program that relies on a set of variables defined by another source file (fancy name: lexical context). So if you ran this template on its own, nothing would happen, because doc isn’t defined. But when you run it in the context of another source file, it picks up the doc that’s defined by that file.

Caution — despite the name, a Pollen template is not necessarily a file of the type suggested by its extension. For instance, fallback.html is a file that ultimately produces HTML, but it’s not actually written in HTML.

It could be, however. Here’s an equivalent way of writing fallback.html that inserts doc into actual HTML, rather than making the whole thing an X-expression.

<html><head><meta charset="UTF-8"></head>
<body>◊(->html doc)</body></html>

Notice that we still need to use the ->html function, but this time, instead of surrounding a larger X-expression, it just goes around doc.

Truly, there is no difference between these two methods. Use whichever works best for you. I often prefer the second method because I like to build & test HTML layouts by hand using placeholder content to make sure all the fiddly bits work. Then it’s easy to replace the placeholder content with (->html doc), and it becomes a template.

6.4.2 Making a custom template

We’ll use these three ingredients to make our own template for article.html.pmd.

In general, template files can have any name you want. But by default, Pollen will first look for a file in your project directory called template.ext, where ext matches the output-file extension of the source file. So if your source file is database.xml.pmd, Pollen will look for template.xml. And for article.html.pmd, Pollen will look for template.html.

Therefore, to set up a custom template, all we need to do is create a file called template.html in our project directory, and make sure it has the three key ingredients we saw in the fallback template. Pollen will automatically apply it to article.html.pmd when we view it in the project server.

But don’t take my word for it. In your project directory, create a new file called template.html:

"template.html"
<html>
<head><meta charset="UTF-8">
<title>Custom template</title></head>
<body>◊(->html doc)</body>
</html>

Recall from the last section that this is the same as the fallback template, but written out in HTML, and with a title element added. In fact, you can now refresh article.html in the project server. Does it look different? No — it won’t, because the resulting template is the same. You should notice, however, that the title of the browser window is now “Custom template,” because Pollen is relying on your new template file, rather than the fallback template.

Let’s change our custom template by adding a style block:

"template.html"
<html>
<head><meta charset="UTF-8">
<title>Custom template</title>
<style type="text/css">
body {padding: 3em; font-size: 20px;}
h1 {background: gray; color: white;}
strong {color: red;}
</style></head>
<body>◊(->html doc)</body>
</html>

When you refresh article.html in the project server, you’ll see that the heading now has a gray background, and one word in the text is red.

Feel free to add other settings to template.html, or update the text in article.html, and see how the page changes. As you’d expect, the project server keeps an eye on both your source files and your template files, and if one changes, it will refresh the output file automatically.

6.4.3 Inserting specific source data into templates

In the last example, we used doc to insert the entire content of the source file — as an X-expression — into the template.

But what if you want to only insert part of your source file into the template? For instance, you’ll look like a dork if the title on each page is “Custom template.” So let’s fix that.

When you’re working in a template, Pollen provides a select function that lets you extract the content of a specific tag, like so: ◊(select ’tag-name doc), which means “get the content of tag-name out of doc and put it here.”

Let’s suppose that we’d rather use the name of the article — Deep Thought — as the page title. We’re going to put a ◊(select ...) command inside the <title> tag.

Beyond that, we just need to know the tag name that contains the title. If we have a little Markdown expertise, we might already know that this part of our Markdown source:

Deep Thought
============

is going to produce a tag named h1.

What if we don’t have all the Markdown conversions memorized? No problem. We can still figure out the tag name by running the article.html.pmd source file in DrRacket and looking at the X-expression that results:

'(root (h1 ((id "my-article")) "Deep Thought") (p () "I am "
(strong () "so") " happy to be writing this."))

Either way, now we know that the text Deep Thought lives in the h1 tag. So we update our template accordingly (for brevity, I’m going to omit the style tag in these examples, but it’s fine to leave it in):

"template.html"
<html>
<head><meta charset="UTF-8">
<title>◊(select 'h1 doc)</title></head>
<body>◊(->html doc)</body>
</html>

When you refresh the page in the project server, the page title will now appear as “Deep Thought.” Of course, you can also combine static and dynamic elements in your template, like so:

"template.html"
<html>
<head><meta charset="UTF-8">
<title>◊(select 'h1 doc), by MB</title></head>
<body>◊(->html doc)</body>
</html>

The page title will now be “Deep Thought, by MB”.

A couple notes on command syntax. We inserted the select and ->html commands using Racket-mode syntax. We could also use text-mode syntax and write the commands this way:

"template.html"
<html>
<head><meta charset="UTF-8">
<title>◊select['h1 doc], by MB</title></head>
<body>◊->html[doc]</body>
</html>

This is exactly equivalent to the previous example. Skeptics are welcome to confirm this by checking the result in the project server.

Finally, notice that in the select command, the tag name 'h1 is written with a quote mark, whereas doc is not. This is an easy place to get tripped up, but the rule is simple: you don’t use a quote mark when you’re referring to the name of an existing function or variable (like select or doc). But you do need a quote mark when you’re using the text as a literal value.

Racket (and hence Pollen) makes a distinction between Symbols (e.g. 'h1) and Strings (e.g. "h1"). Without getting into the weeds, just note for now that the tag of an X-expression is always a symbol, not a string. But if you write ◊(select "h1" doc), the command will still work, because Pollen will treat it as ◊(select 'h1 doc), consistent with a general policy of not being persnickety about input types when the intention is clear.

6.4.4 Linking to an external CSS file

If you’re a super web hotshot, you probably don’t put your CSS selectors in the <head> tag. Instead, you link to an external CSS file. So it will not surprise you that in Pollen, you can do this by adding the usual <link> tag to your HTML template, in this case a file called styles.css:

"template.html"
<html>
<head>
<meta charset="UTF-8">
<title>◊select['h1 doc], by MB</title>
<link rel="stylesheet" type="text/css" media="all" href="styles.css" />
</head>
<body>◊->html[doc]</body>
</html>

Fans of hand-coded CSS, I trust you to take it from here: make your styles.css file, and enjoy the results.

But folks who paid attention during the First tutorial might be wondering “Can we link to a dynamically generated styles.css.pp file?”

Yes, of course. Here’s the rule of thumb: when you’re making links between files — whether CSS, or HTML, or anything else — Pollen doesn’t care whether the file is static or dynamic. You just refer to it by its ultimate name, in this case styles.css. If a static styles.css file exists, Pollen will use that. If it doesn’t, Pollen will look for a source file it can use to make styles.css, and generate it on the spot. (You can also start with a static file, and change it to be dynamic later, and Pollen will do the right thing.)

So to use a dynamic CSS file, we don’t need to make any changes to template.html. We just need to add styles.css.pp to the project directory:

"styles.css.pp"
#lang pollen
 
◊(define h1-color "blue")
◊(define strong-color "green")
 
body {padding: 3em; font-size: 20px;}
h1 {background: ◊|h1-color|; color: white;}
strong {color: ◊|strong-color|;}

Now, when you refresh article.html in the project server, Pollen will generate the styles.css file it needs, and you’ll see the new colors in the page. As before, if you update styles.css.pp, Pollen will notice and regenerate the CSS file when you refresh the page.

Can you add multiple dynamic style sheets? Yes.
Can you mix dynamic and static style sheets? Yes.
Can you add a dynamic JavaScript file? Yes. -
You’re getting the general idea, right? So let’s move on.

6.5 Intermission

If you only need one page for your article, you can stop here. You now know everything necessary to publish a single-page article using authoring mode. You know how to create the mandatory ingredients — a source file and a template — and you also know how to link to an optional CSS file, which can be dynamically generated.

If you want to create a multi-page article, however, you need to get through one more big idea. This might be a good time to take a break.

6.6 Pagetrees

A pagetree is a hierarchical list of Pollen pages. When you have multiple pages in your project, the pagetree establishes relationships among those pages. At its most basic, a pagetree establishes a linear sequence for the pages. But pagetrees can also establish hierarchical relationships — for instance, a book-length project can be organized into chapters, the chapters into sections, and so on. The pagetree doesn’t impose any semantics on the organization of your project. It’s just a tree, and it’s up to you how many layers to establish, what those layers mean, and so on.

Pagemap might’ve been an equally good name, and perhaps more consistent with similar concepts in other web-publishing systems. But I avoided it out of deference to Racket’s map function, which means something completely different.

6.6.1 Pagetree navigation

Pagetrees are used in various ways throughout Pollen. But the most obvious use for a pagetree is to add navigational links to your pages. Obviously, in a multi-page article, readers need a way of getting from one page to the next. In this part of the tutorial, we’ll expand our sample article from one page to three, and see how to create “previous page” and “next page” links in our template that are dynamically generated relative to the current page.

6.6.2 Using the automatic pagetree

You’ve actually already been exposed to pagetrees (though I didn’t tell you about it at the time). Recall that the dashboard of the project server is located at http://localhost:8080/index.ptree. The list of files you see in the dashboard is a pagetree that Pollen creates by reading the files in the current directory and arranging them in alphabetical order.

If the multiple pages in your project are already ordered by filename, then you can rely on this automatic pagetree.

From earlier in the tutorial, you have a Markdown source file called article.html.pmd that looks like this:

"article.html.pmd"
#lang pollen
 
Deep Thought
============
 
I am **so** happy to be writing this.

Let’s supplement this source file by creating two others for the project:

"barticle.html.pmd"
#lang pollen
 
Barticle Title
==============
 
The wonderful second part of the article.
"carticle.html.pmd"
#lang pollen
 
Carticle Title
==============
 
The terrific third part.

As before, you can fill these source files with any sample Markdown content you like. Moreover, you don’t have to use the filenames barticle.html.pmd and carticle.html.pmd — the point is that the intended sequence needs to match the alphabetic sorting of the filenames.

We’ll reuse the template.html and styles.css files from earlier in the tutorial. Move or delete the other tutorial files so that your dashboard in the project server shows only these five files:

  • article.html.pmd

  • barticle.html.pmd

  • carticle.html.pmd

  • styles.css (or styles.css.pp)

  • template.html

If you click on any of the three Markdown sources, you will see it converted into HTML using template.html, with styles appiled from styles.css.

The automatic pagetree for this project is exactly what you see in the dashboard: a list of the three article files, followed by styles.css and template.html.

6.6.3 Adding navigation links to the template with here

Recall from earlier in the tutorial that the content of your source file is made available in the template through the special variable doc. Likewise, the name of the current source file is made available through the special variable here.

To make any navigation link — up, down, sideways — the general idea is that we use here as input to a pagetree-navigation function, which then looks up the answer in the current pagetree.

First, let’s just see here on its own. Update your template as follows:

"template.html"
<html>
<head><meta charset="UTF-8">
<title>◊select['h1 doc], by MB</title></head>
<link rel="stylesheet" type="text/css" media="all" href="styles.css" />
<body>◊->html[doc]
The current page is called ◊|here|.
</body>
</html>

If you refresh article.html, you will now see the line “The current page is called article.html.” Switch to barticle.html, and you’ll see “The current page is called barticle.html.” Makes sense, right?

Now let’s use pagetree functions to show the names of the previous and next pages. Consistent with the usual policy of obviousness, these functions are called previous and next:

"template.html"
<html>
<head><meta charset="UTF-8">
<title>◊select['h1 doc], by MB</title></head>
<link rel="stylesheet" type="text/css" media="all" href="styles.css" />
<body>◊->html[doc]
The current page is called ◊|here|.
The previous is ◊|(previous here)|.
The next is ◊|(next here)|.
</body>
</html>

Refresh barticle.html. You’ll now see that “The current page is called barticle.html. The previous is article.html. The next is carticle.html.” So far, so good: we’re correctly deriving the previous and next pages from the automatic pagetree.

All that’s left is to add hyperlinks, which is easy:

"template.html"
<html>
<head><meta charset="UTF-8">
<title>◊select['h1 doc], by MB</title></head>
<link rel="stylesheet" type="text/css" media="all" href="styles.css" />
<body>◊->html[doc]
The current page is called ◊|here|.
The previous is <a href="◊|(previous here)|">◊|(previous here)|</a>.
The next is <a href="◊|(next here)|">◊|(next here)|</a>.
</body>
</html>

Refresh barticle.html, and you’ll see that the names of the previous and next pages are now hyperlinks to those pages. Click through and convince yourself that it works.

The documentation for pagetree Navigation will tell you about the other functions available for generating navigation links.

6.6.4 Handling navigation boundaries with conditionals

If you clicked through to article.html or carticle.html, you might’ve noticed a couple problems. Because article.html is the first page in the automatic pagetree, it doesn’t have any previous page it can link to. And the next-page link for carticle.html is styles.css, which is strictly correct — it is, in fact, the next file in the automatic pagetree — but it’s not part of our article, so we’d rather stop the navigation there.

One way to fix the problem would be to have three separate template files — the standard one with both previous- and next-page links, one with only a next-page link, and one with only a previous-page link.

But since we have a whole programming language available in Pollen, that’s a dull-witted way to solve the problem. The better way is to add conditionals to the template to selectively change the navigation. That keeps things simple, because we’ll still have only one template.html to deal with.

To handle article.html, we want to hide the previous-page navigation link when there’s no previous page. As it turns out, if the previous function can’t find a previous page, it will return false. So we just need to wrap our previous-page navigation in the when/block command like so:

"template.html"
<html>
<head><meta charset="UTF-8">
<title>◊select['h1 doc], by MB</title></head>
<link rel="stylesheet" type="text/css" media="all" href="styles.css" />
<body>◊->html[doc]
The current page is called ◊|here|.
◊when/block[(previous here)]{The previous is
<a href="◊|(previous here)|">◊|(previous here)|</a>.}
The next is <a href="◊|(next here)|">◊|(next here)|</a>.
</body>
</html>

The basic structure of when/block is ◊when/block[condition]{insert-this-text}. Note the square braces around the condition, and the curly braces around the text. Using (previous here) as the condition is shorthand for “when "(previous here)" does not return false...”

Programmers in the audience might be getting anxious about the repeated use of (previous here) — you’re welcome to store that value in a variable, and everything will work the same way:

"template.html"
<html>
<head><meta charset="UTF-8">
<title>◊select['h1 doc], by MB</title></head>
<link rel="stylesheet" type="text/css" media="all" href="styles.css" />
<body>◊->html[doc]
The current page is called ◊|here|.
◊(define prev-page (previous here))
◊when/block[prev-page]{The previous is
<a href="◊|prev-page|">◊|prev-page|</a>.}
The next is <a href="◊|(next here)|">◊|(next here)|</a>.
</body>
</html>

We need a different technique for handling the end of the next-page navigation, because we’re not reaching the actual end of the pagetree. We’re just reaching the end of the pages we care about navigating through.

What condition will help us detect this? Here, we can notice that the names of our article pages all contain the string article. While you’d probably want a more robust condition for a real project, in this tutorial, what we’ll do is hide the next-page navigation if the name of the next page doesn’t contain “article”. As we did before, we wrap our navigation line in the when/block function:

"template.html"
<html>
<head><meta charset="UTF-8">
<title>◊select['h1 doc], by MB</title></head>
<link rel="stylesheet" type="text/css" media="all" href="styles.css" />
<body>◊->html[doc]
The current page is called ◊|here|.
◊(define prev-page (previous here))
◊when/block[prev-page]{The previous is
<a href="◊|prev-page|">◊|prev-page|</a>.}
◊when/block[(regexp-match "article" (->string (next here)))]{
The next is <a href="◊|(next here)|">◊|(next here)|</a>.}
</body>
</html>

This time, the condition is (regexp-match "article" (->string (next here))). How were you supposed to know this? You weren’t. That’s why this is a tutorial. Without going on a lengthy detour, the regexp-match function returns true if the first string (in this case, "article") is found inside the second string (in this case, we convert (next here) to a string by wrapping it in ->string).

In any case, even if some of the programmy bits went over your head just now, relax and paste the code into your template. What you’ll see when you refresh carticle.html is that the next-page link is gone. So now our template lets us navigate among the pages of our article, and the conditionals handle the end pages correctly.

6.6.5 Making a pagetree file

I didn’t want to dwell on programming complications in the last conditional. Why? The extra programming was necessary only because we made life somewhat difficult for ourselves by relying on the automatic pagetree. A better way to solve the problem is to avoid it altogether by making a pagetree file.

Pagetree source files have a different syntax and status than other Pollen source files, so they are parsed using their own Pollen dialect. To invoke this dialect, you just start the file with #lang pollen and name the file with the ptree extension, for instance my-project.ptree. While you can have as many pagetrees in your project as you want, Pollen will accord primary status to the one named index.ptree.

So let’s make an index.ptree file. At its simplest, a pagetree file can just be a list of files in the intended order. In DrRacket, create a new file in your project directory as follows:

"index.ptree"
#lang pollen
 
carticle.html
article.html
barticle.html

Now run the file. The result will be:

'(pagetree-root carticle.html article.html barticle.html)

Pretty boring, I know. But behind the scenes, Pollen’s pagetree parser is making sure your tree is valid (e.g., no duplicate or malformed names). Today it’s boring, but on the day you have a long and complicated pagetree, you will be grateful.

Notice that the names in this pagetree are the names of output files, not source files. This is deliberate, so that neither you nor Pollen has to care which files are static vs. dynamic. This next pagetree wouldn’t be wrong in the sense of bad syntax — the pagetree parser won’t complain — but it would be wrong in the sense of not-what-you-want, because it refers to source names rather than output names:

"bad-index.ptree"
#lang pollen
 
carticle.html.pmd
article.html.pmd
barticle.html.pmd

You also probably noticed that the files are in a different order than they were in the automatic pagetree: carticle.html is first, followed by article.html and then barticle.html. This too is deliberate, so we can see what happens with a differently ordered pagetree.

Pagetrees don’t change nearly as often as other source files, so as a performance optimization, the project server does not dynamically reflect changes to pagetrees. To see the effect of this new pagetree on our project, you’ll need to go to your terminal window and stop the project server with ctrl+C, and then restart it. Which will take all of three seconds.

Now refresh carticle.html. You’ll notice that the navigation links are different. You won’t see a previous-page link — because carticle.html is now the first page in the pagetree — and the next page will show up as article.html. Click through to article.html, and you’ll see the navigation likewise updated. Click through to barticle.html, and you’ll see ...

BAM! An error page with a yellow box that says Can’t convert #f to string. What happened? We switched to using our own pagetree file but we didn’t update our template conditionals. Once you reach barticle.html, the value of (next here) is false, which means the (->string (next here)) command in the template conditional is trying to convert false into a string. Hence the error.

So let’s go back and fix that. Because we don’t have extraneous files in our pagetree anymore, we can change the second conditional in the template to work the same way as the first:

"template.html"
<html>
<head><meta charset="UTF-8">
<title>◊select['h1 doc], by MB</title></head>
<link rel="stylesheet" type="text/css" media="all" href="styles.css" />
<body>◊->html[doc]
The current page is called ◊|here|.
◊(define prev-page (previous here))
◊when/block[prev-page]{The previous is
<a href="◊|prev-page|">◊|prev-page|</a>.}
◊(define next-page (next here))
◊when/block[next-page]{
The next is <a href="◊|next-page|">◊|next-page|</a>.}
</body>
</html>

Refresh barticle.html — because you’re updating the template, you don’t need to restart the project server — and you’ll see the right result. The previous-page link goes to article.html, and the next-page link is hidden.

6.6.6 index.ptree & the project server

One more thing to show you before we wrap up this tutorial. Remember that the dashboard of the project server is at http://localhost:8080/index.ptree? By default, the project server will synthesize a pagetree from an alphbetical directory listing.

But if you put your own index.ptree file in the directory, the project server will use that for the dashboard instead. In fact, visit http://localhost:8080/index.ptree now and you’ll see what I mean. Consistent with the index.ptree you made, you’ll now see carticle.html, article.html, and barticle.html, but not template.html nor styles.css (even though they’re still in the project directory).

6.7 Second tutorial complete

That was a big tutorial. I commend you for your tenacity and patience. But in this tutorial, you made a giant leap forward. Despite the silly examples, you now know everything you need to make multi-page articles — books, even — using Markdown authoring mode in Pollen. If this is all you ever use Pollen for, it’ll be a big improvement over ordinary Markdown.

But there’s more. We haven’t even gotten into the more elaborate automation that’s possible with Pollen, nor Pollen’s own markup language. We’ll cover that in the third tutorial.

    ← prev  up  next → 
 
\ No newline at end of file +
You’re getting the general idea, right? So let’s move on.

6.5 Intermission

If you only need one page for your article, you can stop here. You now know everything necessary to publish a single-page article using authoring mode. You know how to create the mandatory ingredients — a source file and a template — and you also know how to link to an optional CSS file, which can be dynamically generated.

If you want to create a multi-page article, however, you need to get through one more big idea. This might be a good time to take a break.

6.6 Pagetrees

A pagetree is a hierarchical list of Pollen pages. When you have multiple pages in your project, the pagetree establishes relationships among those pages. At its most basic, a pagetree establishes a linear sequence for the pages. But pagetrees can also establish hierarchical relationships — for instance, a book-length project can be organized into chapters, the chapters into sections, and so on. The pagetree doesn’t impose any semantics on the organization of your project. It’s just a tree, and it’s up to you how many layers to establish, what those layers mean, and so on.

Pagemap might’ve been an equally good name, and perhaps more consistent with similar concepts in other web-publishing systems. But I avoided it out of deference to Racket’s map function, which means something completely different.

6.6.1 Pagetree navigation

Pagetrees are used in various ways throughout Pollen. But the most obvious use for a pagetree is to add navigational links to your pages. Obviously, in a multi-page article, readers need a way of getting from one page to the next. In this part of the tutorial, we’ll expand our sample article from one page to three, and see how to create “previous page” and “next page” links in our template that are dynamically generated relative to the current page.

6.6.2 Using the automatic pagetree

You’ve actually already been exposed to pagetrees (though I didn’t tell you about it at the time). Recall that the dashboard of the project server is located at http://localhost:8080/index.ptree. The list of files you see in the dashboard is a pagetree that Pollen creates by reading the files in the current directory and arranging them in alphabetical order.

If the multiple pages in your project are already ordered by filename, then you can rely on this automatic pagetree.

From earlier in the tutorial, you have a Markdown source file called article.html.pmd that looks like this:

"article.html.pmd"
#lang pollen
 
Deep Thought
============
 
I am **so** happy to be writing this.

Let’s supplement this source file by creating two others for the project:

"barticle.html.pmd"
#lang pollen
 
Barticle Title
==============
 
The wonderful second part of the article.
"carticle.html.pmd"
#lang pollen
 
Carticle Title
==============
 
The terrific third part.

As before, you can fill these source files with any sample Markdown content you like. Moreover, you don’t have to use the filenames barticle.html.pmd and carticle.html.pmd — the point is that the intended sequence needs to match the alphabetic sorting of the filenames.

We’ll reuse the template.html and styles.css files from earlier in the tutorial. Move or delete the other tutorial files so that your dashboard in the project server shows only these five files:

  • article.html.pmd

  • barticle.html.pmd

  • carticle.html.pmd

  • styles.css (or styles.css.pp)

  • template.html

If you click on any of the three Markdown sources, you will see it converted into HTML using template.html, with styles appiled from styles.css.

The automatic pagetree for this project is exactly what you see in the dashboard: a list of the three article files, followed by styles.css and template.html.

6.6.3 Adding navigation links to the template with here

Recall from earlier in the tutorial that the content of your source file is made available in the template through the special variable doc. Likewise, the name of the current source file is made available through the special variable here.

To make any navigation link — up, down, sideways — the general idea is that we use here as input to a pagetree-navigation function, which then looks up the answer in the current pagetree.

First, let’s just see here on its own. Update your template as follows:

"template.html"
<html>
<head>
<meta charset="UTF-8">
<title>◊select['h1 doc], by MB</title>
<link rel="stylesheet" type="text/css" media="all" href="styles.css" />
</head>
<body>◊->html[doc]
The current page is called ◊|here|.
</body>
</html>

If you refresh article.html, you will now see the line “The current page is called article.html.” Switch to barticle.html, and you’ll see “The current page is called barticle.html.” Makes sense, right?

Now let’s use pagetree functions to show the names of the previous and next pages. Consistent with the usual policy of obviousness, these functions are called previous and next:

"template.html"
<html>
<head>
<meta charset="UTF-8">
<title>◊select['h1 doc], by MB</title>
<link rel="stylesheet" type="text/css" media="all" href="styles.css" />
</head>
<body>◊->html[doc]
The current page is called ◊|here|.
The previous is ◊|(previous here)|.
The next is ◊|(next here)|.
</body>
</html>

Refresh barticle.html. You’ll now see that “The current page is called barticle.html. The previous is article.html. The next is carticle.html.” So far, so good: we’re correctly deriving the previous and next pages from the automatic pagetree.

All that’s left is to add hyperlinks, which is easy:

"template.html"
<html>
<head>
<meta charset="UTF-8">
<title>◊select['h1 doc], by MB</title>
<link rel="stylesheet" type="text/css" media="all" href="styles.css" />
</head>
<body>◊->html[doc]
The current page is called ◊|here|.
The previous is <a href="◊|(previous here)|">◊|(previous here)|</a>.
The next is <a href="◊|(next here)|">◊|(next here)|</a>.
</body>
</html>

Refresh barticle.html, and you’ll see that the names of the previous and next pages are now hyperlinks to those pages. Click through and convince yourself that it works.

The documentation for pagetree Navigation will tell you about the other functions available for generating navigation links.

6.6.4 Handling navigation boundaries with conditionals

If you clicked through to article.html or carticle.html, you might’ve noticed a couple problems. Because article.html is the first page in the automatic pagetree, it doesn’t have any previous page it can link to. And the next-page link for carticle.html is styles.css, which is strictly correct — it is, in fact, the next file in the automatic pagetree — but it’s not part of our article, so we’d rather stop the navigation there.

One way to fix the problem would be to have three separate template files — the standard one with both previous- and next-page links, one with only a next-page link, and one with only a previous-page link.

But since we have a whole programming language available in Pollen, that’s a dull-witted way to solve the problem. The better way is to add conditionals to the template to selectively change the navigation. That keeps things simple, because we’ll still have only one template.html to deal with.

To handle article.html, we want to hide the previous-page navigation link when there’s no previous page. As it turns out, if the previous function can’t find a previous page, it will return false. So we just need to wrap our previous-page navigation in the when/block command like so:

"template.html"
<html>
<head>
<meta charset="UTF-8">
<title>◊select['h1 doc], by MB</title>
<link rel="stylesheet" type="text/css" media="all" href="styles.css" />
</head>
<body>◊->html[doc]
The current page is called ◊|here|.
◊when/block[(previous here)]{The previous is
<a href="◊|(previous here)|">◊|(previous here)|</a>.}
The next is <a href="◊|(next here)|">◊|(next here)|</a>.
</body>
</html>

The basic structure of when/block is ◊when/block[condition]{insert-this-text}. Note the square braces around the condition, and the curly braces around the text. Using (previous here) as the condition is shorthand for “when "(previous here)" does not return false...”

Programmers in the audience might be getting anxious about the repeated use of (previous here) — you’re welcome to store that value in a variable, and everything will work the same way:

"template.html"
<html>
<head>
<meta charset="UTF-8">
<title>◊select['h1 doc], by MB</title>
<link rel="stylesheet" type="text/css" media="all" href="styles.css" />
</head>
<body>◊->html[doc]
The current page is called ◊|here|.
◊(define prev-page (previous here))
◊when/block[prev-page]{The previous is
<a href="◊|prev-page|">◊|prev-page|</a>.}
The next is <a href="◊|(next here)|">◊|(next here)|</a>.
</body>
</html>

We need a different technique for handling the end of the next-page navigation, because we’re not reaching the actual end of the pagetree. We’re just reaching the end of the pages we care about navigating through.

What condition will help us detect this? Here, we can notice that the names of our article pages all contain the string article. While you’d probably want a more robust condition for a real project, in this tutorial, what we’ll do is hide the next-page navigation if the name of the next page doesn’t contain “article”. As we did before, we wrap our navigation line in the when/block function:

"template.html"
<html>
<head>
<meta charset="UTF-8">
<title>◊select['h1 doc], by MB</title>
<link rel="stylesheet" type="text/css" media="all" href="styles.css" />
</head>
<body>◊->html[doc]
The current page is called ◊|here|.
◊(define prev-page (previous here))
◊when/block[prev-page]{The previous is
<a href="◊|prev-page|">◊|prev-page|</a>.}
◊when/block[(regexp-match "article" (->string (next here)))]{
The next is <a href="◊|(next here)|">◊|(next here)|</a>.}
</body>
</html>

This time, the condition is (regexp-match "article" (->string (next here))). How were you supposed to know this? You weren’t. That’s why this is a tutorial. Without going on a lengthy detour, the regexp-match function returns true if the first string (in this case, "article") is found inside the second string (in this case, we convert (next here) to a string by wrapping it in ->string).

In any case, even if some of the programmy bits went over your head just now, relax and paste the code into your template. What you’ll see when you refresh carticle.html is that the next-page link is gone. So now our template lets us navigate among the pages of our article, and the conditionals handle the end pages correctly.

6.6.5 Making a pagetree file

I didn’t want to dwell on programming complications in the last conditional. Why? The extra programming was necessary only because we made life somewhat difficult for ourselves by relying on the automatic pagetree. A better way to solve the problem is to avoid it altogether by making a pagetree file.

Pagetree source files have a different syntax and status than other Pollen source files, so they are parsed using their own Pollen dialect. To invoke this dialect, you just start the file with #lang pollen and name the file with the ptree extension, for instance my-project.ptree. While you can have as many pagetrees in your project as you want, Pollen will accord primary status to the one named index.ptree.

So let’s make an index.ptree file. At its simplest, a pagetree file can just be a list of files in the intended order. In DrRacket, create a new file in your project directory as follows:

"index.ptree"
#lang pollen
 
carticle.html
article.html
barticle.html

Now run the file. The result will be:

'(pagetree-root carticle.html article.html barticle.html)

Pretty boring, I know. But behind the scenes, Pollen’s pagetree parser is making sure your tree is valid (e.g., no duplicate or malformed names). Today it’s boring, but on the day you have a long and complicated pagetree, you will be grateful.

Notice that the names in this pagetree are the names of output files, not source files. This is deliberate, so that neither you nor Pollen has to care which files are static vs. dynamic. This next pagetree wouldn’t be wrong in the sense of bad syntax — the pagetree parser won’t complain — but it would be wrong in the sense of not-what-you-want, because it refers to source names rather than output names:

"bad-index.ptree"
#lang pollen
 
carticle.html.pmd
article.html.pmd
barticle.html.pmd

You also probably noticed that the files are in a different order than they were in the automatic pagetree: carticle.html is first, followed by article.html and then barticle.html. This too is deliberate, so we can see what happens with a differently ordered pagetree.

Pagetrees don’t change nearly as often as other source files, so as a performance optimization, the project server does not dynamically reflect changes to pagetrees. To see the effect of this new pagetree on our project, you’ll need to go to your terminal window and stop the project server with ctrl+C, and then restart it. Which will take all of three seconds.

Now refresh carticle.html. You’ll notice that the navigation links are different. You won’t see a previous-page link — because carticle.html is now the first page in the pagetree — and the next page will show up as article.html. Click through to article.html, and you’ll see the navigation likewise updated. Click through to barticle.html, and you’ll see ...

BAM! An error page with a yellow box that says Can’t convert #f to string. What happened? We switched to using our own pagetree file but we didn’t update our template conditionals. Once you reach barticle.html, the value of (next here) is false, which means the (->string (next here)) command in the template conditional is trying to convert false into a string. Hence the error.

So let’s go back and fix that. Because we don’t have extraneous files in our pagetree anymore, we can change the second conditional in the template to work the same way as the first:

"template.html"
<html>
<head>
<meta charset="UTF-8">
<title>◊select['h1 doc], by MB</title>
<link rel="stylesheet" type="text/css" media="all" href="styles.css" />
</head>
<body>◊->html[doc]
The current page is called ◊|here|.
◊(define prev-page (previous here))
◊when/block[prev-page]{The previous is
<a href="◊|prev-page|">◊|prev-page|</a>.}
◊(define next-page (next here))
◊when/block[next-page]{
The next is <a href="◊|next-page|">◊|next-page|</a>.}
</body>
</html>

Refresh barticle.html — because you’re updating the template, you don’t need to restart the project server — and you’ll see the right result. The previous-page link goes to article.html, and the next-page link is hidden.

6.6.6 index.ptree & the project server

One more thing to show you before we wrap up this tutorial. Remember that the dashboard of the project server is at http://localhost:8080/index.ptree? By default, the project server will synthesize a pagetree from an alphbetical directory listing.

But if you put your own index.ptree file in the directory, the project server will use that for the dashboard instead. In fact, visit http://localhost:8080/index.ptree now and you’ll see what I mean. Consistent with the index.ptree you made, you’ll now see carticle.html, article.html, and barticle.html, but not template.html nor styles.css (even though they’re still in the project directory).

6.7 Second tutorial complete

That was a big tutorial. I commend you for your tenacity and patience. But in this tutorial, you made a giant leap forward. Despite the silly examples, you now know everything you need to make multi-page articles — books, even — using Markdown authoring mode in Pollen. If this is all you ever use Pollen for, it’ll be a big improvement over ordinary Markdown.

But there’s more. We haven’t even gotten into the more elaborate automation that’s possible with Pollen, nor Pollen’s own markup language. We’ll cover that in the third tutorial.

    ← prev  up  next → 
 
\ No newline at end of file diff --git a/doc/third-tutorial.html b/doc/third-tutorial.html index 6283c0e..69e2892 100644 --- a/doc/third-tutorial.html +++ b/doc/third-tutorial.html @@ -1,9 +1,3 @@ -7 Third tutorial
▼Pollen:   the book is a program
►7 Third tutorial
On this page:
7.1 Prerequisites
7.2 Pollen markup vs. XML
7.2.1 The XML problem
7.2.2 What Pollen markup does differently
7.2.3 “But I really need XML…”
7.3 Writing with Pollen markup
7.3.1 Creating a Pollen markup file
7.3.2 Tags & tag functions
7.3.3 Attributes
7.3.4 What are custom tags good for?
7.3.5 Semantic markup
7.3.6 Format independence
7.3.7 Using custom tags
7.3.8 Choosing custom tags
7.4 Tags are functions
7.4.1 Attaching behavior to tags
7.4.2 Notes for experienced programmers
7.4.2.1 Point of no return
7.4.2.2 Multiple input values & rest arguments
7.4.2.3 Returning an X-expression
7.4.2.4 Interpolating variables into strings
7.4.2.5 Parsing attributes
7.5 Intermission
7.6 Organizing functions
7.6.1 Using Racket’s function libraries
7.6.2 The directory-require.rkt file
7.7 Decoding markup via the root tag
7.8 Putting it all together
6.1.0.5
    ← prev  up  next → 

7 Third tutorial

Now you’re getting to the good stuff. In this tutorial, you’ll use Pollen to publish a multi-page article written in Pollen markup. You’ll learn about:

  • Adding tags & attributes with Pollen markup

  • Attaching behavior to tag functions

  • the directory-require.rkt file

  • Using decode with Pollen markup

  • raco pollen render and raco pollen clone

If you want the shortest possible introduction to Pollen, try the Quick tour.

7.1 Prerequisites

I’ll assume you’ve completed the Second tutorial and that you understand the principles of Pollen authoring mode — creating source files, converting them to X-expressions, and then combining them with templates to make output files.

Because now it’s time to pick up the pace. You’ve learned how to do some handy things with Pollen. But we haven’t yet exploited the full fusion of writing environment and programming language. I promised you that The book is a program, right? So let’s do some programming.

7.2 Pollen markup vs. XML

You can skip this section if XML holds no interest. But Pollen markup evolved out of my attempt to come up with an alternative to XML that would be more usable for writing. So if you’re familiar with XML, the contrast may be helpful.

7.2.1 The XML problem

In the Second tutorial, I made the case that Markdown is a limiting format for authors. Why? Markdown is essentially a notation system for HTML tags. As such, it has three problems: it’s not semantic, it only covers a limited subset of HTML tags, and it can’t be extended by an author.

These problems are partly limitations of HTML itself. And these limitations were meant to be cured by XML — the X stands for extensible. In principle, XML allows you to define whatever tags you like and use them in your document.

So why hasn’t XML taken over the world? In practice, XML promises more than it delivers. The reasons are apparent to any writer who’s attempted to use XML as an authoring format:

  • Verbose syntax. Unfortunately, XML relies on the same angle-bracket notation as HTML. If you think HTML source is hard to read, XML is even worse. Since much of writing involves reading, this feature is also a major bug.

  • Validation overhead. Integral to XML is the concept of validation, which guarantees that a document meets certain formal criteria, usually asserted in a schema. To get the full value from XML, you generally want to use validation. But doing so imposes a lot more work on you as an author, and removes much of the expressive potential of XML.

  • Masochistic document processing. I’m referring to XSLT, the preferred method of transforming XML documents. I know a little XSLT, so I’ll concede that there’s a method to its madness. But it’s still madness.

The nicest thing we could say about XML is that its intentions are good. It’s oriented toward the right goals. But its benefits are buried under atrocious ergonomics.

7.2.2 What Pollen markup does differently

Pollen markup can be seen as a way of reaping the benefits of XML without incurring the headaches. Like XML, Pollen markup allows you to freely tag your text. But unlike XML:

  • Simple syntax. Pollen markup follows the usual conventions of Pollen commands.

  • No structural validation. You can use any tags you want, in any order, and you needn’t define them ahead of time. Your document will still work.

  • Racket processing. Pollen markup tags can have behavior attached to them using Racket functions, either before you use them, or later.

7.2.3 “But I really need XML…”

You can have XML. There’s nothing wrong with using Pollen markup to generate XML files that can then be fed into an existing XML processing pipeline. In other words, using Pollen markup, you can treat XML as an output format rather than an input format.

In this tutorial, I’ll be rendering Pollen markup with an HTML template. But you could easily use the same workflow with an XML template and thus end up with XML files.

7.3 Writing with Pollen markup

Pollen markup is a free-form markup system that lets you add arbitrary tags and attributes to your text. By arbitrary, I mean that they don’t need to match up with an existing schema or specification (e.g., the tags permitted by HTML). They can — but that’s an option, not a requirement.

I like to think of Pollen markup a way of capturing not just the text, but also my ideas about the text. Some of these are low-level ideas (“this text should be italicized”). Some are high-level ideas (“this text is the topic of the page”). Some are just notes to myself. In short, everything I know about the text becomes part of the text.

In so doing, Pollen markup becomes the source code of the book. Let’s try it out.

7.3.1 Creating a Pollen markup file

We’re going to use Pollen markup to make a file that will ultimately be HTML. So consistent with the authoring-mode workflow we learned in the Second tutorial, we’ll start with our desired output filename, article.html, and then append the Pollen markup suffix, .pm.

In DrRacket, start a new file called article.html.pm like so (BTW you can use any sample text you like):

"article.html.pm"
#lang pollen
 
I want to attend RacketCon this year.

Consistent with usual authoring-mode policy, when you run this file, you’ll get an X-expression that starts with root:

'(root "I want to attend RacketCon this year.")

Remember, even though the first line of the file is #lang pollen — same as the last tutorial — the new .pm suffix signals that Pollen should interpret the source as Pollen markup. Look what happens if you goof up and put Markdown source in a Pollen markup file, like so:

#lang pollen
 
I am **so** excited to attend __RacketCon__ this year.

The Markdown syntax will be ignored, and pass through to the output:

'(root "I am **so** excited to attend __RacketCon__ this year.")

Restore the non-Markdown source, and let’s continue.

7.3.2 Tags & tag functions

Pollen markup uses the same Pollen command syntax that we first saw in Adding commands. Previously, we used this command syntax to invoke functions like define and ->html. Pollen markup is used to invoke a special kind of function called a tag function, which is a function that, by default, adds a tag to the text.

To see how this works, restore your article.html.pm file to its original state:

#lang pollen
 
I want to attend RacketCon this year.

We can add any tag with Pollen markup, but for now, let’s start with an old favorite: em, which is used in HTML to add emphasis to text. We apply a tag by starting with the lozenge character (◊) followed by the tag name em, followed by the text in curly braces, like so:

"article.html.pm"
#lang pollen
 
I want to attend ◊em{RacketCon this year}.

Run this file in DrRacket and see the X-expression that results:

'(root "I want to attend " (em "RacketCon this year") ".")

You won’t be surprised to hear that you can nest tags:

"article.html.pm"
#lang pollen
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

With the expected results:

'(root "I want to attend " (em "RacketCon " (strong "this") " year") ".")

7.3.3 Attributes

Attributes are like tags for tags. Each attribute is a key–value pair where the key is any name, and the value is a string. Anyone who’s seen HTML is familiar with them:

<span class="author">Prof. Leonard</span>

Here, class is an attribute for span that has value "author". And this is what it looks like as an X-expression:

'(span ((class "author")) "Prof. Leonard")

You can add any number of attributes to a tag (first as an X-expression, then as HTML):

'(span ((class "author")(id "primary")(living "true")) "Prof. Leonard")

<span class="author" id="primary" living="true">Prof. Leonard</span>

In Pollen markup, attributes have the same logic, but a slightly different syntax. In keeping with the tag notation you just saw, the span tag is added in the usual way:

"article.html.pm"
#lang pollen
 
◊span{Prof. Leonard}

Then you have two options for adding attributes. The verbose way corresponds to how the attributes appear in the X-expression:

"article.html.pm"
#lang pollen
 
◊span['((class "author")(id "primary")(living "true"))]{Prof. Leonard}

Each key–value pair is in parentheses, and then the list of pairs is within parentheses, with a quote (') at the front that signals that the text should be used literally.

This involves some superfluous typing, however, so Pollen also supports an abbreviated syntax for attributes:

"article.html.pm"
#lang pollen
 
◊span['class:"author" 'id:"primary" 'living:"true"]{Prof. Leonard}

In this form, each attribute key starts with a quote mark ' and ends with a colon :. As before, the attribute value is in quotation marks.

Both of these forms will produce the same X-expression:

'(span ((class "author")(id "primary")(living "true")) "Prof. Leonard")

Now that you know how to make tags and attributes, you might wonder whether Pollen markup can be used as a quick & dirty HTML-notation system. Sure — for a quick & dirty project, why not. Recall that X-expressions are just alternative notation for the standard angle-bracket notation used in HTML. So if you wanted HTML like this:

<div class="red" style="font-size:150%">Important <em>News</em></div>

You could write it in Pollen markup like so:

◊div['class:"red" style:"font-size:150%"]{Important ◊em{News}}

And then just convert it (using the ->html function) into the HTML above. Thus, the tags you already know and love (?) can be used in Pollen markup, but with fewer keystrokes and cruft.

Still, if Pollen markup were just an alternative notation system for HTML tags, it would be pretty boring. As I alluded above, that’s merely a boring way to use it.

In the XML spirit, Pollen markup lets you use any tags you want. That’s considerably less boring.

7.3.4 What are custom tags good for?

XML jocks can skip this section, since you already know. But if you’ve been mired in Markdown or HTML, read on.

Tags, broadly speaking, are a means of annotating a text with extra information, which I’ll call metadata (using that term in its generic sense, not in any fiddly computery way). Metadata is the key tool that enables an author to write a book with the benefits of semantic markup and format independence.

7.3.5 Semantic markup

Semantic markup means adding metadata to text according to the meaning of the text, not merely its intended visual appearance. So rather than tagging RacketCon with an em tag, as we did above to indicate how the word should look, maybe we would tag it with an event tag, to indicate what kind of thing it is.

Semantic markup lets an author specify distinctions that would be ambiguous in pure visual terms, thereby capturing more meaning and intent. For instance, in books, italic styling is commonly applied to a number of unrelated types of information: emphasized words, movie titles, terms being used for the first time, headings, captions and labels, and so on. Under a non-semantic formatting scheme, perhaps one would tag them all em. But in semantic terms, one would tag them movie-title, first-use, heading, as appropriate.

This has two major benefits. First, by separating appearance and meaning, an author can manage the content of the book in useful ways. For instance, if every movie title were tagged as movie-title rather than italic, then it would be simple to generate a list of all movies mentioned in the book (for the author’s benefit) or a page index of movie references (for the reader’s benefit). But without that semantic tagging, a movie title couldn’t be distinguished from any other italicized text.

7.3.6 Format independence

The second benefit of custom tags is format independence, or the ability to change the rendering of the text to suit a particular device or context.

When a text is encrusted with format-specific visual tags — for instance, HTML tags — then the document markup is entangled with a single output format. If you only need one output format, fine.

But increasingly, book authors have been called upon to publish their work in multiple formats: paper and PDF, but also web, e-book, or other natively digital formats, that connect to devices with differing display capabilities.

Yes, I know that many of these formats are based on variants of HTML. But the HTML you can use in a desktop web browser is quite different from, say, the HTML you can use in a Kindle .mobi file. The .mobi file has other technical requirements too, like an .ncx and .opf file. So despite some genetic kinship, these HTML-ish formats are best understood as separate targets.

Using a display-driven model to manage this complexity is a terrible idea — as anyone who’s tried it can attest. Converting from one display-based file type to another — for instance, word processor to HTML, or HTML to PDF — is an exercise in frustration and drain-circling expectations.

This isn’t surprising. For a long time, text processing has been dominated by this display-driven model. Most word processors, like Microsoft Word and Pages, have been built around this model. It worked well enough in the era where most documents were eventually going to be printed on paper (or a paper simulator like PDF). HTML was a technical leap forward, but not a conceptual leap: it mostly represented the display options available in a web browser.

There’s a couple TeX fans at the back of the room, waving their arms. Yes, TeX got a lot of things right. In practice, however, it never became a core tool for electronic publishing (which, to be fair, didn’t exist when TeX was written). Plenty of ideas in Pollen were lifted from TeX.

For a document to be format independent, two conditions have to be satisfied.

First, the document has to be readable by other programs, so they can handle the conversion of format-independent markup into a format-specific rendering (e.g., mapping semantic tags like movie-title onto visual tags like em). Most word-processor formats, like Word’s .docx, are bad for authoring because these formats are opaque and proprietary. We needn’t get into the political objections. As a practical matter, they’re inarguably restrictive — if you can’t get your data out of your file, you’re stuck.

Second, the document itself has to be represented in a way that’s independent of the particularities of any one format. For instance, HTML is a bad authoring format because it encourages authors to litter their text with HTML-isms like h1 and span. These have no meaning outside of HTML, and thus will always cause conversion problems. The same goes for Markdown, which is simply HTML in disguise.

The solution to the first condition is to use text-based markup rather than proprietary file types. The solution to the second condition is to let authors define custom tags for the document, rather than the other way around. Pollen markup incorporates both of these ideas.

7.3.7 Using custom tags

You can insert a custom tag using the same syntax as any other tag. Suppose you want to use an event tag to mark events. You would insert it like so:

"article.html.pm"
#lang pollen
 
I want to attend ◊event{RacketCon} this year.

This markup will turn into this X-expression:

'(root "I want to attend " (event "RacketCon") " this year.")

Which is equivalent to this XML:

<root>I want to attend <event>RacketCon</event> this year.</root>

In truth, Pollen doesn’t notice any difference between a custom tag vs. a standard HTML tag vs. any other kind of tag. They’re all just markup tags. If you want to restrict yourself to a certain vocabulary of tags, you can. If you want to set up Pollen to enforce those restrictions, you can do that too. But by default, Pollen doesn’t impose restrictions like this. In general, you can pick any tag name you want, and it will work.

Don’t take my word for it. See what happens if you write this:

"article.html.pm"
#lang pollen
 
I want to attend ◊verylongandimpracticaltagname{RacketCon} this year.

One small but important exception to this rule. If you were wondering why I sometimes call them tag functions instead of just tags, it’s because under the hood, every tag is implemented as a function. The default behavior of this function is just to wrap the text in a tag with the given name.

The benefit of treating tags as functions will become evident later in this tutorial. But the cost of this approach is that tags occupy the same namespace as the other functions available in Pollen (and by extension, Racket). So if you try to use a tag name that’s already the name of an existing function, an error will occur.

For instance, let’s suppose you try to use a custom tag called length:

"article.html.pm"
#lang pollen
 
The Panama Canal is ◊length{77km} across.

When you run this file, you get an error:

length: contract violation;
expected: list?
  given: "77km"

The problem is that Racket already provides a function called length. Consistent with the usual rules of Pollen command notation, your command is interpreted as an attempt to invoke the length function, rather than apply a tag named length.

In practice, namespace clashes are rare. But if necessary, they’re easy to work around (for the simplest method, see Invoking tag functions).

7.3.8 Choosing custom tags

You just saw that using custom tags is easy. Choosing custom tags, on the other hand, is less science than art. As the author, it’s up to you. Some guidelines:

  • You’re never doing it wrong. I wanted to make sure you knew the case for semantic markup. But if your life would be easier just using HTML tags directly, go ahead.

  • Tag iteratively. Don’t worry about getting all your tags right the first time through. Just as you write and then rewrite, add the tags that seem right now, and change or augment them later, because …

  • Tags emerge from writing. It’s hopeless to try to specify all your tags in advance. As you write, you’ll learn things about the text, which will suggest new tags.

  • The best tag system is the one you’ll stick with. Tags aren’t free. It takes effort to insert them consistently. Don’t bother with an overambitious tag scheme that bores you more than it helps.

  • For boilerplate, tags are faster than text. If you find yourself repeatedly formatting certain text in a certain way — for instance, lists and tables — extract the content and wrap it in a tag that encapsulates the boilerplate.

And most important:

  • Tags are functions. As I mentioned above, every tag has a function behind it that uses the content of the tag as input. The default tag function just outputs the tag and its content. But you can replace this with any kind of function. So in practice, you can offload a lot of labor to tags.

As we’ll see in the next section, this is where your book truly becomes programmable.

7.4 Tags are functions

Don’t skip this section! It explains a concept that’s essential to understanding how Pollen works.

If you’ve used HTML or XML, tags are just tags: things you type into the document that look the same going out as they did going in. Tags can be used to select document elements or assign styling (via CSS). But they don’t have any deeper effect on the document content.

That’s not so in Pollen. Under the hood, Pollen is just an alternate way of writing code in the Racket programming language. And tags, instead of being inert markers, are actually functions.

I think most of you know what a function is, but just to be safe — in programming, a function is a chunk of code that accepts some input, processes it, and then returns a value. Asking a function to process some data is known as calling the function.

Leading us to the Three Golden Rules of Pollen Tags:

  1. Every Pollen tag calls a function with the same name.

  2. The input values for that function are the attributes and content of the tag.

  3. The whole tag — tag name, attributes, and content — is replaced with the return value of the called function.

You’ve already seen the simplest kind of function in a Pollen document: the default tag function, which emulates the behavior of standard markup tags.

Let’s revisit an earlier example, now with the help of the Golden Rules:

"article.html.pm"
#lang pollen
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

What happens when you run this source? Working from the inside out, Pollen calls the function strong with the input "this". The result is (strong "this"). Then Pollen calls the function em with the three input values "RacketCon " (strong "this") " year", which yields (em "RacketCon " (strong "this") " year"). Finally, Pollen calls the root function with everything in the document, resulting in:

'(root "I want to attend " (em "RacketCon " (strong "this") " year") ".")

7.4.1 Attaching behavior to tags

Sometimes this default behavior will suffice. But other times, you’ll want to change the behavior of a tag. Why? Here are some useful examples of what you, as an author, can do with custom tag functions:

  • Automatically detect cross-references and add hyperlinks.

  • Pull in data from an external source.

  • Generate tables, figures, and other fiddly layout objects.

  • Change content based on given conditions.

  • Automatically detect line breaks, paragraphs, and lists.

  • Insert boilerplate text.

  • Anything annoying or repetitive.

  • Mathematical computations.

  • … and anything else you like to do with a programming language.

Having invited you to gaze across these vistas, I apologize that my example here in this tutorial is necessarily tip-of-the-iceberg. I’ll be adding a more detailed guide to writing Pollen functions, both simple and crafty.

How do you change the behavior of a tag? By 1) writing a new function and 2) giving it the name of the tag. Once you do this, this new behavior will automatically be invoked when you use the tag.

For example, let’s redefine the strong tag in our example above to simply print BOOM:

"article.html.pm"
#lang pollen
 
◊define[(strong . lines)]{BOOM}
 
I want to attend ◊em{RacketCon ◊strong{this} year}

When you run this file, you indeed get:

'(root "I want to attend " (em "RacketCon " "BOOM" " year"))

How does this work? First, although you can define a function in Pollen command syntax using either of The two command modes: text mode & Racket mode, it tends to be easier to use Racket mode. I wrote the first one in text mode. But for clarity, I’m going to switch to Racket mode (run this file and convince yourself it comes out the same):

"article.html.pm"
#lang pollen
 
◊(define (strong word) "BOOM")
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

Let’s look at our new function definition. As usual, we start with the lozenge character (◊) to denote a Pollen command. Then we use define to introduce a function definition. The name of the function comes next, which needs to match our tag name, strong. The expression (strong word) means “the name of this function is strong, and it takes a single word as input, which we’ll refer to as word.” Finally we have the return value, which is "BOOM".

Let’s run this file again, but go back to the Golden Rules to understand what happens. Working from the inside out, Pollen calls the function strong with the input "this" — same as before. But this time, the result of the strong function is not (strong "this"), but simply BOOM. Then Pollen calls the function em with the three input values "RacketCon " "BOOM" " year", which yields (em "RacketCon " "BOOM" " year"). Finally, Pollen calls the root function with everything in the document, resulting in:

'(root "I want to attend " (em "RacketCon " "BOOM" " year"))

This example is contrived, of course. But the basic idea — defining a function with the name of a tag — is the foundation of programmability in Pollen. If you get this, and the Golden Rules, you get everything.

7.4.2 Notes for experienced programmers

Having said that, some of you are probably eager to hack around a bit. Let me chip off a few more cubes from the iceberg to help you on your way. (Everyone else, take five.)

7.4.2.1 Point of no return

If you’ve written functions in other programming languages, you might be accustomed to using a return statement to send a value back from the function. This doesn’t exist in Pollen or Racket — the return value of any function is just the last expression evaluated. In the example below, "BAP" becomes the return value because it’s in the last position, and "BOOM" is ignored:

"article.html.pm"
#lang pollen
 
◊(define (strong word) "BOOM" "BAP")
 
I want to attend ◊em{RacketCon ◊strong{this} year}.
7.4.2.2 Multiple input values & rest arguments

Sometimes a tag will have only one word or string that becomes its input. More likely, however, it will have multiple values (this is inevitable with nested tags, because the results aren’t concatenated). For instance, if we attach our function to em rather than strong:

"article.html.pm"
#lang pollen
 
◊(define (em word) "BOOM")
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

Look what happens:

em: arity mismatch;
the expected number of arguments does not match the given number
expected: 1
  given: 3

The error arises because the em function is getting three arguments — "RacketCon " "BOOM" " year" — but has been defined to only accept one argument, word. This is the “arity mismatch.”

To fix this, it’s better to get in the habit of writing tag functions that accept an indefinite number of input values. You do this by defining your function with a rest argument (as in, “give me the rest of the input values.”) To use a rest argument, put it last in your list of input arguments, and add a period . before:

"article.html.pm"
#lang pollen
 
◊(define (em . parts) "BOOM")
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

This time, the source file will run without an error, producing this:

'(root "I want to attend " "BOOM" ".")

A rest argument like parts is a list of individual arguments. So if you want to unpack & process these arguments separately, you can use Racket’s extensive list-processing functions (see Pairs and Lists). Also see quasiquote below.

7.4.2.3 Returning an X-expression

Often, you won’t use a tag function to replace a whole tag with a string — you’ll replace it with a different tag, described by an X-expression, like so:

"article.html.pm"
#lang pollen
 
◊(define (em . parts) '(big "BOOM"))
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

Which produces:

'(root "I want to attend " (big "BOOM") ".")

The quote mark ' before the X-expression signals to Racket that you want to use what follows as a literal value.

To build X-expressions that are more elaborate, you have two options.

First is quasiquote. Quasiquote works like quote, but starts with a backtick character `. What makes it “quasi” is that you can insert variables using the unquote operator, which is a comma , or merge a list of values with the unquote-splicing operator, which is a comma followed by an @ sign ,@.

Let’s adapt the example above to use quasiquote. Suppose we want to take the parts we get as input and put them inside a big tag. This is easy to notate with quasiquote and the unquote-splicing operator, because parts is a list:

"article.html.pm"
#lang pollen
 
◊(define (em . parts) `(big ,@parts))
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

Which produces this:

'(root "I want to attend " (big "RacketCon " (strong "this") " year") ".")

Of course you can also nest X-expressions in your return value:

"article.html.pm"
#lang pollen
 
◊(define (em . parts) `(extra (big ,@parts)))
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

The second option for building X-expressions is to use the txexpr: Tagged X-expressions library that’s included with Pollen (see those docs for more information).

7.4.2.4 Interpolating variables into strings

The usual way is to use the format function:

(format "String with variable: ~a" variable-name)

See the docs for format and fprintf for your options.

Be careful if you’re working with integers and X-expressions — a raw integer is treated as a character code, not an integer string. Using format is essential:

Examples:

> (->html '(div "A raw integer indicates a character code: " 42))

"<div>A raw integer indicates a character code: &#42;</div>"

> (->html `(div "Use format to make it a string: " ,(format "~a" 42)))

"<div>Use format to make it a string: 42</div>"

7.4.2.5 Parsing attributes

Detecting attributes in an argument list can be tricky because a) the tag may or may not have attributes, b) those attributes may be in standard or abbreviated syntax. For this reason, Pollen provides a split-attributes function (in the pollen/tag librar) that you can use in custom tag functions to separate the attributes and elements:

"article.html.pm"
#lang pollen
 
◊(require pollen/tag)
 
◊(define (em . parts)
  (define-values (attributes elements) (split-attributes parts))
  `(extra ,attributes (big ,@elements)))
 
I want to attend ◊em['key: "value"]{RacketCon}.

This will move the elements inside the big tag, and attach the attributes to the extra tag:

'(root "I want to attend " (extra ((key "value")) (big "RacketCon")) ".")

7.5 Intermission

That was a lot of heavy material. But it also covered the most essential idea in Pollen: that every tag is a function. Congratulations on making it this far.

The good news is that the rest of this tutorial will feel more relaxed, as we put these new principles to work.

Sorry that this tutorial is longer than the others, but truly — this is the stuff that makes Pollen different. If you’re not feeling enthusiastic by now, you should bail out.

Otherwise, get ready to rock.

7.6 Organizing functions

In the tag-function examples so far, we’ve defined each function within the source file where we used it. This is fine for quick little functions.

But more often, you’re going to want to use functions defined elsewhere, and store your own functions available so they’re available to your source files.

For now, we’re just invoking functions within a Pollen markup file. But as you’ll see in the fourth tutorial, any function can be called from any kind of Pollen source file.

7.6.1 Using Racket’s function libraries

Any function in Racket’s extensive libraries can be called by loading the library with the require command, which will make all its functions and constants available with the usual Pollen command syntax:

"article.html.pm"
#lang pollen
 
◊(require racket/math)
 
Pi is close to ◊|pi|.
The hyperbolic sine of pi is close to ◊(sinh pi).

The result:

'(root "Pi is close to " 3.141592653589793 "." "\n" "The hyperbolic sine of pi is close to " 11.548739357257748 ".")

One caveat — you’re still in a Pollen markup file, so the return value of whatever function you call has to produce a string or an X-expression, so it can be merged into the document. This is similar to the restriction introduced in the first tutorial where functions used in preprocessor files had to produce text. -Pollen won’t stop you from calling a function that returns an incompatible value, like plot, which returns a bitmap image:

"article.html.pm"
#lang pollen
 
◊(require math plot)
 
Here's a sine wave:
◊(plot (function sin (- pi) pi #:label "y = sin(x)"))

But it won’t work when you try to run it in DrRacket or load it in the project server.

It would be fine, however, to call a different kind of plot function that returned an SVG result, because any XML-ish data structure can be converted to an X-expression.

Super web nerds also know that binary data can be converted into XML-ish form by encoding the file as a base-64 data URL — but if you know what I’m talking about, then you don’t need my help to try it.

For functions that don’t return a string or an X-expression, you can always make a conversion by hand. For instance, consider range, a Racket function that returns a list of integers:

"article.html.pm"
#lang pollen
 
◊(require racket/list)
A list of integers: ◊(range 5)

This will produce an error in DrRacket:

pollen markup error: in '(root "A list of integers: " (0 1 2 3 4)), '(0 1 2 3 4) is not a valid element (must be txexpr, string, symbol, XML char, or cdata)

In a case like this, you can explicitly convert the return value to a string (in whatever way makes sense):

"article.html.pm"
#lang pollen
 
◊(require racket/list racket/string)
A list of integers: ◊(string-join (map number->string (range 5)))

And get this output:

'(root "A list of integers: " "0 1 2 3 4")

7.6.2 The directory-require.rkt file

Don’t skip this section! It explains a concept that’s essential to understanding how Pollen works.

As you get more comfortable attaching behavior to tags using tag functions, you’ll likely want to create some functions that can be shared between multiple source files. The directory-require.rkt file is a special file that is automatically imported by Pollen source files in the same directory. So every function and value provided by directory-require.rkt can be used in these Pollen files.

First, using this file is not mandatory. You can always import functions and values from another file using require (as seen in the previous section). The directory-require.rkt is just meant to cure the tedium of importing the same file into every Pollen source file in your project. In a small project, not much tedium; in a large project, more.

Second, notice from the .rkt suffix that directory-require.rkt is a source file containing Racket code, not Pollen code. This is the default because while Pollen is better for text-driven source files, Racket is better for code-driven source files. Still, the choice is yours: the name of this file can be changed by resetting the world:directory-require value.

Third, notice from the directory- prefix that directory-require.rkt is only used by Pollen source files in the same directory. So if your project has source files nested inside a subdirectory, you’ll need to explicitly create another directory-require.rkt there and share the functions & values as needed.

“Why not make this file visible throughout a project, rather than just a directory?” Good idea, but I couldn’t figure out how to do it without creating finicky new dependencies. If you have a better idea, I’m open to it.

Let’s see how this works in practice. In the same directory as article.html.pm, create a new directory-require.rkt file as follows:

"directory-require.rkt"
#lang racket
(define author "Trevor Goodchild")
(provide author)

Here we use the define function (which we’ve seen before) to set author equal to "Trevor Goodchild". Note the final step: consistent with standard Racket rules, we have to explicitly provide the new value so that other files can see it (unlike Python, things you define in Racket are private by default, not public).

Then update good old article.html.pm:

"article.html.pm"
#lang pollen
 
The author is ◊|author|.

Run this in DrRacket and you’ll get:

'(root "The author is " "Trevor Goodchild" ".")

Now, in the same dirctory, create a second Pollen source file:

"barticle.html.pm"
#lang pollen
 
The author is really ◊|author|?

Run this, and you’ll get:

'(root "The author is really " "Trevor Goodchild" "?")

That’s all there is to it. Everything provided by directory-require.rkt is automatically available within each Pollen source file.

You can include functions, including tag functions, the same way. For instance, add a function for em:

"directory-require.rkt"
#lang racket
(define author "Trevor Goodchild")
(define (em . parts) `(extra (big ,@parts)))
(provide author em)

Then use it in a source file:

"article.html.pm"
#lang pollen
 
The ◊em{author} is ◊em{◊|author|}.

With the expected results:

'(root "The " (extra (big "author")) " is " (extra (big "Trevor Goodchild")) ".")

7.7 Decoding markup via the root tag

As you’ve seen, the X-expression you get when you run a Pollen markup file always starts with a node called root. You can attach a tag function to root the same way as any other tag. For instance, you could do something simple, like change the name of the output X-expression:

"article.html.pm"
#lang pollen
 
◊(define (root . elements) `(content ,@elements))
 
The ◊tt{root} tag is now called ◊tt{content}.

Resulting in:

'(content "The " (tt "root") " tag is now called " (tt "content") ".")

But unlike other tags in your document, root contains the entire content of the document. So the function you attach to root can operate on everything.

For that reason, one of the most useful things you can do with a tag function attached to root is decoding the content of the page. Decoding refers to any post-processing of content that happens after the tags within the page have been evaluated.

Decoding is a good way to automatically accomplish:

  • Detection of linebreaks, paragraphs, and list items based on whitespace.

  • Hyphenation.

  • Typographic optimizations, like smart quotes, dashes, and ligatures.

  • Gathering data for indexing or cross-referencing.

  • Any document enhancements a) that can be handled programmatically and b) that you’d prefer not to hard-code within your source files.

As an example, let’s take one of my favorites — linebreak and paragraph detection. In XML authoring, you have to insert every <br /> and <p> tag by hand. This is profoundly dull, clutters up the source file, and makes editing a chore.

Instead, let’s make a decoder that allows us to denote a linebreak with a single newline in the source, and a paragraph break with a double newline. Here’s some sample content with single and double newlines:

"article.html.pm"
#lang pollen
 
The first line of the first paragraph.
And a new line.
 
The second paragraph.

But without a decoder, the newlines just get passed through:

'(root "The first line of the first paragraph." "\n" "And a new line." "\n" "\n" "The second paragraph.")

When this X-expression is converted to HTML, the newlines persist:

<root>The first line of the first paragraph.\nAnd a new line.\n\nThe second paragraph.</root>

But in HTML, raw newlines are displayed as a single space. So if you view this file in the project server, you’ll see:

The first line of the first paragraph. And a new line. The second paragraph.

Not what we want.

So we need to make a decoder. To do this, we use the decode-elements function, which provides hooks to selectively process certain categories of content within the document.

decode-elements is a convenience variant of decode, which takes a full X-expression as input. Under the hood, they work the same way, so use whichever you prefer.

Add a basic decode-elements to the source file like so:

"article.html.pm"
#lang pollen
 
◊(require pollen/decode txexpr)
◊(define (root . elements)
   (make-txexpr 'root null (decode-elements elements)))
 
The first line of the first paragraph.
And a new line.
 
The second paragraph.

The make-txexpr function is a utility from the txexpr package, which is installed with Pollen. It builds a new X-expression from a tag, attribute list, and list of elements. Here, we’ll keep the tag name root, leave the attributes as null, and append our decoded list of elements.

Racket jocks: you could also write this using quasiquote and unquote-splicing syntax as `(root ,@(decode-elements elements)). The txexpr package is just a more explicit way of accomplishing the task.

If you run this file, what changes? Right — nothing. That’s because by default, both decode-elements (and decode) will let the content pass through unaltered.

We change this by giving decode-elements the name of a processing function and attaching it to the type of content we want to process. In this case, we’re in luck — the decode module already contains a detect-paragraphs function (that also detects linebreaks). We add this function using the keyword argument #:txexpr-elements-proc, which is short for “the function used to process the elements of a tagged X-expression”:

"article.html.pm"
#lang pollen
 
◊(require pollen/decode txexpr)
◊(define (root . elements)
   (make-txexpr 'root null (decode-elements elements
     #:txexpr-elements-proc detect-paragraphs)))
 
The first line of the first paragraph.
And a new line.
 
The second paragraph.

Now, when we run the file, the X-expression has changed to include two p tags and a br tag:

'(root (p "The first line of the first paragraph." (br) "And a new line.") (p "The second paragraph."))

That means when we convert to HTML, we get the tags we need:

<root><p>The first line of the first paragraph.<br />And a new line.</p><p>The second paragraph.</p></root>

So when we view this in the project server, the linebreaks and paragraph breaks are displayed correctly:

The first line of the first paragraph.
-And a new line. -

-The second paragraph.

Of course, in practice you wouldn’t put your decoding function in a single source file. You’d make it available to all your source files by putting it in directory-require.rkt. So let’s do that now:

"directory-require.rkt"
#lang racket
(require pollen/decode txexpr)
(define (root . elements)
   (make-txexpr 'root null (decode-elements elements
     #:txexpr-elements-proc detect-paragraphs)))
(provide (all-defined-out))

We’ll also restore the source of article.html.pm to its original, simplified state:

"article.html.pm"
#lang pollen
 
The first line of the first paragraph.
And a new line.
 
The second paragraph.

And the result in the project server will be the same:

The first line of the first paragraph.
-And a new line. -

-The second paragraph.

By the way, though decoding via the root tag is the most likely usage scenario, you don’t have to do it that way. Decoding is just a special kind of tag function. So you can make a decoder that only affects a certain tag within the page. Or you can make multiple decoders for different tags. The advantage of using a decoder with root is that it can affect all the content, and it will be the last tag function that gets called.

7.8 Putting it all together

[Coming soon]

    ← prev  up  next → 
 
\ No newline at end of file +7 Third tutorial
▼Pollen:   the book is a program
►7 Third tutorial
On this page:
7.1 Prerequisites
7.2 Pollen markup vs. XML
7.2.1 The XML problem
7.2.2 What Pollen markup does differently
7.2.3 “But I really need XML…”
7.3 Writing with Pollen markup
7.3.1 Creating a Pollen markup file
7.3.2 Tags & tag functions
7.3.3 Attributes
7.3.4 What are custom tags good for?
7.3.5 Semantic markup
7.3.6 Format independence
7.3.7 Using custom tags
7.3.8 Choosing custom tags
7.4 Tags are functions
7.4.1 Attaching behavior to tags
7.4.2 Notes for experienced programmers
7.4.2.1 Point of no return
7.4.2.2 Multiple input values & rest arguments
7.4.2.3 Returning an X-expression
7.4.2.4 Interpolating variables into strings
7.4.2.5 Parsing attributes
7.5 Intermission
7.6 Organizing functions
7.6.1 Using Racket’s function libraries
7.6.2 Using the directory-require.rkt file
7.7 Decoding markup via the root tag
7.8 Putting it all together
7.8.1 The directory-require.rkt file
7.8.2 The template
7.8.3 The pagetree
7.8.4 A CSS stylesheet using the preprocessor
7.8.5 The content source files using Pollen markup
7.8.6 The result
7.9 Third tutorial complete
6.1.0.5
    ← prev  up  next → 

7 Third tutorial

Now you’re getting to the good stuff. In this tutorial, you’ll use Pollen to publish a multi-page article written in Pollen markup. You’ll learn about:

  • Adding tags & attributes with Pollen markup

  • Attaching behavior to tag functions

  • the directory-require.rkt file

  • Using decode with Pollen markup

If you want the shortest possible introduction to Pollen, try the Quick tour.

7.1 Prerequisites

I’ll assume you’ve completed the Second tutorial and that you understand the principles of Pollen authoring mode — creating source files, converting them to X-expressions, and then combining them with templates to make output files.

Because now it’s time to pick up the pace. You’ve learned how to do some handy things with Pollen. But we haven’t yet exploited the full fusion of writing environment and programming language. I promised you that The book is a program, right? So let’s do some programming.

7.2 Pollen markup vs. XML

You can skip this section if XML holds no interest. But Pollen markup evolved out of my attempt to come up with an alternative to XML that would be more usable for writing. So if you’re familiar with XML, the contrast may be helpful.

7.2.1 The XML problem

In the Second tutorial, I made the case that Markdown is a limiting format for authors. Why? Markdown is essentially a notation system for HTML tags. As such, it has three problems: it’s not semantic, it only covers a limited subset of HTML tags, and it can’t be extended by an author.

These problems are partly limitations of HTML itself. And these limitations were meant to be cured by XML — the X stands for extensible. In principle, XML allows you to define whatever tags you like and use them in your document.

So why hasn’t XML taken over the world? In practice, XML promises more than it delivers. The reasons are apparent to any writer who’s attempted to use XML as an authoring format:

  • Verbose syntax. Unfortunately, XML relies on the same angle-bracket notation as HTML. If you think HTML source is hard to read, XML is even worse. Since much of writing involves reading, this feature is also a major bug.

  • Validation overhead. Integral to XML is the concept of validation, which guarantees that a document meets certain formal criteria, usually asserted in a schema. To get the full value from XML, you generally want to use validation. But doing so imposes a lot more work on you as an author, and removes much of the expressive potential of XML.

  • Masochistic document processing. I’m referring to XSLT, the preferred method of transforming XML documents. I know a little XSLT, so I’ll concede that there’s a method to its madness. But it’s still madness.

The nicest thing we could say about XML is that its intentions are good. It’s oriented toward the right goals. But its benefits are buried under atrocious ergonomics.

7.2.2 What Pollen markup does differently

Pollen markup can be seen as a way of reaping the benefits of XML without incurring the headaches. Like XML, Pollen markup allows you to freely tag your text. But unlike XML:

  • Simple syntax. Pollen markup follows the usual conventions of Pollen commands.

  • No structural validation. You can use any tags you want, in any order, and you needn’t define them ahead of time. Your document will still work.

  • Racket processing. Pollen markup tags can have behavior attached to them using Racket functions, either before you use them, or later.

7.2.3 “But I really need XML…”

You can have XML. There’s nothing wrong with using Pollen markup to generate XML files that can then be fed into an existing XML processing pipeline. In other words, using Pollen markup, you can treat XML as an output format rather than an input format.

In this tutorial, I’ll be rendering Pollen markup with an HTML template. But you could easily use the same workflow with an XML template and thus end up with XML files.

7.3 Writing with Pollen markup

Pollen markup is a free-form markup system that lets you add arbitrary tags and attributes to your text. By arbitrary, I mean that they don’t need to match up with an existing schema or specification (e.g., the tags permitted by HTML). They can — but that’s an option, not a requirement.

I like to think of Pollen markup a way of capturing not just the text, but also my ideas about the text. Some of these are low-level ideas (“this text should be italicized”). Some are high-level ideas (“this text is the topic of the page”). Some are just notes to myself. In short, everything I know about the text becomes part of the text.

In so doing, Pollen markup becomes the source code of the book. Let’s try it out.

7.3.1 Creating a Pollen markup file

We’re going to use Pollen markup to make a file that will ultimately be HTML. So consistent with the authoring-mode workflow we learned in the Second tutorial, we’ll start with our desired output filename, article.html, and then append the Pollen markup suffix, .pm.

In DrRacket, start a new file called article.html.pm like so (BTW you can use any sample text you like):

"article.html.pm"
#lang pollen
 
I want to attend RacketCon this year.

Consistent with usual authoring-mode policy, when you run this file, you’ll get an X-expression that starts with root:

'(root "I want to attend RacketCon this year.")

Remember, even though the first line of the file is #lang pollen — same as the last tutorial — the new .pm suffix signals that Pollen should interpret the source as Pollen markup. Look what happens if you goof up and put Markdown source in a Pollen markup file, like so:

#lang pollen
 
I am **so** excited to attend __RacketCon__ this year.

The Markdown syntax will be ignored, and pass through to the output:

'(root "I am **so** excited to attend __RacketCon__ this year.")

Restore the non-Markdown source, and let’s continue.

7.3.2 Tags & tag functions

Pollen markup uses the same Pollen command syntax that we first saw in Adding commands. Previously, we used this command syntax to invoke functions like define and ->html. Pollen markup is used to invoke a special kind of function called a tag function, which is a function that, by default, adds a tag to the text.

To see how this works, restore your article.html.pm file to its original state:

#lang pollen
 
I want to attend RacketCon this year.

We can add any tag with Pollen markup, but for now, let’s start with an old favorite: em, which is used in HTML to add emphasis to text. We apply a tag by starting with the lozenge character (◊) followed by the tag name em, followed by the text in curly braces, like so:

"article.html.pm"
#lang pollen
 
I want to attend ◊em{RacketCon this year}.

Run this file in DrRacket and see the X-expression that results:

'(root "I want to attend " (em "RacketCon this year") ".")

You won’t be surprised to hear that you can nest tags:

"article.html.pm"
#lang pollen
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

With the expected results:

'(root "I want to attend " (em "RacketCon " (strong "this") " year") ".")

7.3.3 Attributes

Attributes are like tags for tags. Each attribute is a key–value pair where the key is any name, and the value is a string. Anyone who’s seen HTML is familiar with them:

<span class="author">Prof. Leonard</span>

Here, class is an attribute for span that has value "author". And this is what it looks like as an X-expression:

'(span ((class "author")) "Prof. Leonard")

You can add any number of attributes to a tag (first as an X-expression, then as HTML):

'(span ((class "author")(id "primary")(living "true")) "Prof. Leonard")

<span class="author" id="primary" living="true">Prof. Leonard</span>

In Pollen markup, attributes have the same logic, but a slightly different syntax. In keeping with the tag notation you just saw, the span tag is added in the usual way:

"article.html.pm"
#lang pollen
 
◊span{Prof. Leonard}

Then you have two options for adding attributes. The verbose way corresponds to how the attributes appear in the X-expression:

"article.html.pm"
#lang pollen
 
◊span['((class "author")(id "primary")(living "true"))]{Prof. Leonard}

Each key–value pair is in parentheses, and then the list of pairs is within parentheses, with a quote (') at the front that signals that the text should be used literally.

This involves some superfluous typing, however, so Pollen also supports an abbreviated syntax for attributes:

"article.html.pm"
#lang pollen
 
◊span['class:"author" 'id:"primary" 'living:"true"]{Prof. Leonard}

In this form, each attribute key starts with a quote mark ' and ends with a colon :. As before, the attribute value is in quotation marks.

Both of these forms will produce the same X-expression:

'(span ((class "author")(id "primary")(living "true")) "Prof. Leonard")

Now that you know how to make tags and attributes, you might wonder whether Pollen markup can be used as a quick & dirty HTML-notation system. Sure — for a quick & dirty project, why not. Recall that X-expressions are just alternative notation for the standard angle-bracket notation used in HTML. So if you wanted HTML like this:

<div class="red" style="font-size:150%">Important <em>News</em></div>

You could write it in Pollen markup like so:

◊div['class:"red" style:"font-size:150%"]{Important ◊em{News}}

And then just convert it (using the ->html function) into the HTML above. Thus, the tags you already know and love (?) can be used in Pollen markup, but with fewer keystrokes and cruft.

Still, if Pollen markup were just an alternative notation system for HTML tags, it would be pretty boring. As I alluded above, that’s merely a boring way to use it.

In the XML spirit, Pollen markup lets you use any tags you want. That’s considerably less boring.

7.3.4 What are custom tags good for?

XML jocks can skip this section, since you already know. But if you’ve been mired in Markdown or HTML, read on.

Tags, broadly speaking, are a means of annotating a text with extra information, which I’ll call metadata (using that term in its generic sense, not in any fiddly computery way). Metadata is the key tool that enables an author to write a book with the benefits of semantic markup and format independence.

7.3.5 Semantic markup

Semantic markup means adding metadata to text according to the meaning of the text, not merely its intended visual appearance. So rather than tagging RacketCon with an em tag, as we did above to indicate how the word should look, maybe we would tag it with an event tag, to indicate what kind of thing it is.

Semantic markup lets an author specify distinctions that would be ambiguous in pure visual terms, thereby capturing more meaning and intent. For instance, in books, italic styling is commonly applied to a number of unrelated types of information: emphasized words, movie titles, terms being used for the first time, headings, captions and labels, and so on. Under a non-semantic formatting scheme, perhaps one would tag them all em. But in semantic terms, one would tag them movie-title, first-use, heading, as appropriate.

This has two major benefits. First, by separating appearance and meaning, an author can manage the content of the book in useful ways. For instance, if every movie title were tagged as movie-title rather than italic, then it would be simple to generate a list of all movies mentioned in the book (for the author’s benefit) or a page index of movie references (for the reader’s benefit). But without that semantic tagging, a movie title couldn’t be distinguished from any other italicized text.

7.3.6 Format independence

The second benefit of custom tags is format independence, or the ability to change the rendering of the text to suit a particular device or context.

When a text is encrusted with format-specific visual tags — for instance, HTML tags — then the document markup is entangled with a single output format. If you only need one output format, fine.

But increasingly, book authors have been called upon to publish their work in multiple formats: paper and PDF, but also web, e-book, or other natively digital formats, that connect to devices with differing display capabilities.

Yes, I know that many of these formats are based on variants of HTML. But the HTML you can use in a desktop web browser is quite different from, say, the HTML you can use in a Kindle .mobi file. The .mobi file has other technical requirements too, like an .ncx and .opf file. So despite some genetic kinship, these HTML-ish formats are best understood as separate targets.

Using a display-driven model to manage this complexity is a terrible idea — as anyone who’s tried it can attest. Converting from one display-based file type to another — for instance, word processor to HTML, or HTML to PDF — is an exercise in frustration and drain-circling expectations.

This isn’t surprising. For a long time, text processing has been dominated by this display-driven model. Most word processors, like Microsoft Word and Pages, have been built around this model. It worked well enough in the era where most documents were eventually going to be printed on paper (or a paper simulator like PDF). HTML was a technical leap forward, but not a conceptual leap: it mostly represented the display options available in a web browser.

There’s a couple TeX fans at the back of the room, waving their arms. Yes, TeX got a lot of things right. In practice, however, it never became a core tool for electronic publishing (which, to be fair, didn’t exist when TeX was written). Plenty of ideas in Pollen were lifted from TeX.

For a document to be format independent, two conditions have to be satisfied.

First, the document has to be readable by other programs, so they can handle the conversion of format-independent markup into a format-specific rendering (e.g., mapping semantic tags like movie-title onto visual tags like em). Most word-processor formats, like Word’s .docx, are bad for authoring because these formats are opaque and proprietary. We needn’t get into the political objections. As a practical matter, they’re inarguably restrictive — if you can’t get your data out of your file, you’re stuck.

Second, the document itself has to be represented in a way that’s independent of the particularities of any one format. For instance, HTML is a bad authoring format because it encourages authors to litter their text with HTML-isms like h1 and span. These have no meaning outside of HTML, and thus will always cause conversion problems. The same goes for Markdown, which is simply HTML in disguise.

The solution to the first condition is to use text-based markup rather than proprietary file types. The solution to the second condition is to let authors define custom tags for the document, rather than the other way around. Pollen markup incorporates both of these ideas.

7.3.7 Using custom tags

You can insert a custom tag using the same syntax as any other tag. Suppose you want to use an event tag to mark events. You would insert it like so:

"article.html.pm"
#lang pollen
 
I want to attend ◊event{RacketCon} this year.

This markup will turn into this X-expression:

'(root "I want to attend " (event "RacketCon") " this year.")

Which is equivalent to this XML:

<root>I want to attend <event>RacketCon</event> this year.</root>

In truth, Pollen doesn’t notice any difference between a custom tag vs. a standard HTML tag vs. any other kind of tag. They’re all just markup tags. If you want to restrict yourself to a certain vocabulary of tags, you can. If you want to set up Pollen to enforce those restrictions, you can do that too. But by default, Pollen doesn’t impose restrictions like this. In general, you can pick any tag name you want, and it will work.

Don’t take my word for it. See what happens if you write this:

"article.html.pm"
#lang pollen
 
I want to attend ◊verylongandimpracticaltagname{RacketCon} this year.

One small but important exception to this rule. If you were wondering why I sometimes call them tag functions instead of just tags, it’s because under the hood, every tag is implemented as a function. The default behavior of this function is just to wrap the text in a tag with the given name.

The benefit of treating tags as functions will become evident later in this tutorial. But the cost of this approach is that tags occupy the same namespace as the other functions available in Pollen (and by extension, Racket). So if you try to use a tag name that’s already the name of an existing function, an error will occur.

For instance, let’s suppose you try to use a custom tag called length:

"article.html.pm"
#lang pollen
 
The Panama Canal is ◊length{77km} across.

When you run this file, you get an error:

length: contract violation;
expected: list?
  given: "77km"

The problem is that Racket already provides a function called length. Consistent with the usual rules of Pollen command notation, your command is interpreted as an attempt to invoke the length function, rather than apply a tag named length.

In practice, namespace clashes are rare. But if necessary, they’re easy to work around (for the simplest method, see Invoking tag functions).

7.3.8 Choosing custom tags

You just saw that using custom tags is easy. Choosing custom tags, on the other hand, is less science than art. As the author, it’s up to you. Some guidelines:

  • You’re never doing it wrong. I wanted to make sure you knew the case for semantic markup. But if your life would be easier just using HTML tags directly, go ahead.

  • Tag iteratively. Don’t worry about getting all your tags right the first time through. Just as you write and then rewrite, add the tags that seem right now, and change or augment them later, because …

  • Tags emerge from writing. It’s hopeless to try to specify all your tags in advance. As you write, you’ll learn things about the text, which will suggest new tags.

  • The best tag system is the one you’ll stick with. Tags aren’t free. It takes effort to insert them consistently. Don’t bother with an overambitious tag scheme that bores you more than it helps.

  • For boilerplate, tags are faster than text. If you find yourself repeatedly formatting certain text in a certain way — for instance, lists and tables — extract the content and wrap it in a tag that encapsulates the boilerplate.

And most important:

  • Tags are functions. As I mentioned above, every tag has a function behind it that uses the content of the tag as input. The default tag function just outputs the tag and its content. But you can replace this with any kind of function. So in practice, you can offload a lot of labor to tags.

As we’ll see in the next section, this is where your book truly becomes programmable.

7.4 Tags are functions

Don’t skip this section! It explains a concept that’s essential to understanding how Pollen works.

If you’ve used HTML or XML, tags are just tags: things you type into the document that look the same going out as they did going in. Tags can be used to select document elements or assign styling (via CSS). But they don’t have any deeper effect on the document content.

That’s not so in Pollen. Under the hood, Pollen is just an alternate way of writing code in the Racket programming language. And tags, instead of being inert markers, are actually functions.

I think most of you know what a function is, but just to be safe — in programming, a function is a chunk of code that accepts some input, processes it, and then returns a value. Asking a function to process some data is known as calling the function.

Leading us to the Three Golden Rules of Pollen Tags:

  1. Every Pollen tag calls a function with the same name.

  2. The input values for that function are the attributes and content of the tag.

  3. The whole tag — tag name, attributes, and content — is replaced with the return value of the called function.

You’ve already seen the simplest kind of function in a Pollen document: the default tag function, which emulates the behavior of standard markup tags.

Let’s revisit an earlier example, now with the help of the Golden Rules:

"article.html.pm"
#lang pollen
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

What happens when you run this source? Working from the inside out, Pollen calls the function strong with the input "this". The result is (strong "this"). Then Pollen calls the function em with the three input values "RacketCon " (strong "this") " year", which yields (em "RacketCon " (strong "this") " year"). Finally, Pollen calls the root function with everything in the document, resulting in:

'(root "I want to attend " (em "RacketCon " (strong "this") " year") ".")

7.4.1 Attaching behavior to tags

Sometimes this default behavior will suffice. But other times, you’ll want to change the behavior of a tag. Why? Here are some useful examples of what you, as an author, can do with custom tag functions:

  • Automatically detect cross-references and add hyperlinks.

  • Pull in data from an external source.

  • Generate tables, figures, and other fiddly layout objects.

  • Change content based on given conditions.

  • Automatically detect line breaks, paragraphs, and lists.

  • Insert boilerplate text.

  • Anything annoying or repetitive.

  • Mathematical computations.

  • … and anything else you like to do with a programming language.

Having invited you to gaze across these vistas, I apologize that my example here in this tutorial is necessarily tip-of-the-iceberg. I’ll be adding a more detailed guide to writing Pollen functions, both simple and crafty.

How do you change the behavior of a tag? By 1) writing a new function and 2) giving it the name of the tag. Once you do this, this new behavior will automatically be invoked when you use the tag.

For example, let’s redefine the strong tag in our example above to simply print BOOM:

"article.html.pm"
#lang pollen
 
◊define[(strong . lines)]{BOOM}
 
I want to attend ◊em{RacketCon ◊strong{this} year}

When you run this file, you indeed get:

'(root "I want to attend " (em "RacketCon " "BOOM" " year"))

How does this work? First, although you can define a function in Pollen command syntax using either of The two command modes: text mode & Racket mode, it tends to be easier to use Racket mode. I wrote the first one in text mode. But for clarity, I’m going to switch to Racket mode (run this file and convince yourself it comes out the same):

"article.html.pm"
#lang pollen
 
◊(define (strong word) "BOOM")
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

Let’s look at our new function definition. As usual, we start with the lozenge character (◊) to denote a Pollen command. Then we use define to introduce a function definition. The name of the function comes next, which needs to match our tag name, strong. The expression (strong word) means “the name of this function is strong, and it takes a single word as input, which we’ll refer to as word.” Finally we have the return value, which is "BOOM".

Let’s run this file again, but go back to the Golden Rules to understand what happens. Working from the inside out, Pollen calls the function strong with the input "this" — same as before. But this time, the result of the strong function is not (strong "this"), but simply BOOM. Then Pollen calls the function em with the three input values "RacketCon " "BOOM" " year", which yields (em "RacketCon " "BOOM" " year"). Finally, Pollen calls the root function with everything in the document, resulting in:

'(root "I want to attend " (em "RacketCon " "BOOM" " year"))

This example is contrived, of course. But the basic idea — defining a function with the name of a tag — is the foundation of programmability in Pollen. If you get this, and the Golden Rules, you get everything.

7.4.2 Notes for experienced programmers

Having said that, some of you are probably eager to hack around a bit. Let me chip off a few more cubes from the iceberg to help you on your way. (Everyone else, take five.)

7.4.2.1 Point of no return

If you’ve written functions in other programming languages, you might be accustomed to using a return statement to send a value back from the function. This doesn’t exist in Pollen or Racket — the return value of any function is just the last expression evaluated. In the example below, "BAP" becomes the return value because it’s in the last position, and "BOOM" is ignored:

"article.html.pm"
#lang pollen
 
◊(define (strong word) "BOOM" "BAP")
 
I want to attend ◊em{RacketCon ◊strong{this} year}.
7.4.2.2 Multiple input values & rest arguments

Sometimes a tag will have only one word or string that becomes its input. More likely, however, it will have multiple values (this is inevitable with nested tags, because the results aren’t concatenated). For instance, if we attach our function to em rather than strong:

"article.html.pm"
#lang pollen
 
◊(define (em word) "BOOM")
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

Look what happens:

em: arity mismatch;
the expected number of arguments does not match the given number
expected: 1
  given: 3

The error arises because the em function is getting three arguments — "RacketCon " "BOOM" " year" — but has been defined to only accept one argument, word. This is the “arity mismatch.”

To fix this, it’s better to get in the habit of writing tag functions that accept an indefinite number of input values. You do this by defining your function with a rest argument (as in, “give me the rest of the input values.”) To use a rest argument, put it last in your list of input arguments, and add a period . before:

"article.html.pm"
#lang pollen
 
◊(define (em . parts) "BOOM")
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

This time, the source file will run without an error, producing this:

'(root "I want to attend " "BOOM" ".")

A rest argument like parts is a list of individual arguments. So if you want to unpack & process these arguments separately, you can use Racket’s extensive list-processing functions (see Pairs and Lists). Also see quasiquote below.

7.4.2.3 Returning an X-expression

Often, you won’t use a tag function to replace a whole tag with a string — you’ll replace it with a different tag, described by an X-expression, like so:

"article.html.pm"
#lang pollen
 
◊(define (em . parts) '(big "BOOM"))
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

Which produces:

'(root "I want to attend " (big "BOOM") ".")

The quote mark ' before the X-expression signals to Racket that you want to use what follows as a literal value.

To build X-expressions that are more elaborate, you have two options.

First is quasiquote. Quasiquote works like quote, but starts with a backtick character `. What makes it “quasi” is that you can insert variables using the unquote operator, which is a comma , or merge a list of values with the unquote-splicing operator, which is a comma followed by an @ sign ,@.

Let’s adapt the example above to use quasiquote. Suppose we want to take the parts we get as input and put them inside a big tag. This is easy to notate with quasiquote and the unquote-splicing operator, because parts is a list:

"article.html.pm"
#lang pollen
 
◊(define (em . parts) `(big ,@parts))
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

Which produces this:

'(root "I want to attend " (big "RacketCon " (strong "this") " year") ".")

Of course you can also nest X-expressions in your return value:

"article.html.pm"
#lang pollen
 
◊(define (em . parts) `(extra (big ,@parts)))
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

The second option for building X-expressions is to use the txexpr: Tagged X-expressions library that’s included with Pollen (see those docs for more information).

7.4.2.4 Interpolating variables into strings

The usual way is to use the format function:

(format "String with variable: ~a" variable-name)

See the docs for format and fprintf for your options.

Be careful if you’re working with integers and X-expressions — a raw integer is treated as a character code, not an integer string. Using format is essential:

Examples:

> (->html '(div "A raw integer indicates a character code: " 42))

"<div>A raw integer indicates a character code: &#42;</div>"

> (->html `(div "Use format to make it a string: " ,(format "~a" 42)))

"<div>Use format to make it a string: 42</div>"

7.4.2.5 Parsing attributes

Detecting attributes in an argument list can be tricky because a) the tag may or may not have attributes, b) those attributes may be in standard or abbreviated syntax. For this reason, Pollen provides a split-attributes function (in the pollen/tag librar) that you can use in custom tag functions to separate the attributes and elements:

"article.html.pm"
#lang pollen
 
◊(require pollen/tag)
 
◊(define (em . parts)
  (define-values (attributes elements) (split-attributes parts))
  `(extra ,attributes (big ,@elements)))
 
I want to attend ◊em['key: "value"]{RacketCon}.

This will move the elements inside the big tag, and attach the attributes to the extra tag:

'(root "I want to attend " (extra ((key "value")) (big "RacketCon")) ".")

7.5 Intermission

That was a lot of heavy material. But it also covered the most essential idea in Pollen: that every tag is a function. Congratulations on making it this far.

The good news is that the rest of this tutorial will feel more relaxed, as we put these new principles to work.

Sorry that this tutorial is longer than the others, but truly — this is the stuff that makes Pollen different. If you’re not feeling enthusiastic by now, you should bail out.

Otherwise, get ready to rock.

7.6 Organizing functions

In the tag-function examples so far, we’ve defined each function within the source file where we used it. This is fine for quick little functions.

But more often, you’re going to want to use functions defined elsewhere, and store your own functions available so they’re available to your source files.

For now, we’re just invoking functions within a Pollen markup file. But as you’ll see in the fourth tutorial, any function can be called from any kind of Pollen source file.

7.6.1 Using Racket’s function libraries

Any function in Racket’s extensive libraries can be called by loading the library with the require command, which will make all its functions and constants available with the usual Pollen command syntax:

"article.html.pm"
#lang pollen
 
◊(require racket/math)
 
Pi is close to ◊|pi|.
The hyperbolic sine of pi is close to ◊(sinh pi).

The result:

'(root "Pi is close to " 3.141592653589793 "." "\n" "The hyperbolic sine of pi is close to " 11.548739357257748 ".")

One caveat — you’re still in a Pollen markup file, so the return value of whatever function you call has to produce a string or an X-expression, so it can be merged into the document. This is similar to the restriction introduced in the first tutorial where functions used in preprocessor files had to produce text. +Pollen won’t stop you from calling a function that returns an incompatible value, like plot, which returns a bitmap image:

"article.html.pm"
#lang pollen
 
◊(require math plot)
 
Here's a sine wave:
◊(plot (function sin (- pi) pi #:label "y = sin(x)"))

But it won’t work when you try to run it in DrRacket or load it in the project server.

It would be fine, however, to call a different kind of plot function that returned an SVG result, because any XML-ish data structure can be converted to an X-expression.

Super web nerds also know that binary data can be converted into XML-ish form by encoding the file as a base64 data URL — but if you know what I’m talking about, then you don’t need my help to try it.

For functions that don’t return a string or an X-expression, you can always make a conversion by hand. For instance, consider range, a Racket function that returns a list of integers:

"article.html.pm"
#lang pollen
 
◊(require racket/list)
A list of integers: ◊(range 5)

This will produce an error in DrRacket:

pollen markup error: in '(root "A list of integers: " (0 1 2 3 4)), '(0 1 2 3 4) is not a valid element (must be txexpr, string, symbol, XML char, or cdata)

In a case like this, you can explicitly convert the return value to a string (in whatever way makes sense):

"article.html.pm"
#lang pollen
 
◊(require racket/list racket/string)
A list of integers: ◊(string-join (map number->string (range 5)))

And get this output:

'(root "A list of integers: " "0 1 2 3 4")

7.6.2 Using the directory-require.rkt file

Don’t skip this section! It explains a concept that’s essential to understanding how Pollen works.

As you get more comfortable attaching behavior to tags using tag functions, you’ll likely want to create some functions that can be shared between multiple source files. The directory-require.rkt file is a special file that is automatically imported by Pollen source files in the same directory. So every function and value provided by directory-require.rkt can be used in these Pollen files.

First, using this file is not mandatory. You can always import functions and values from another file using require (as seen in the previous section). The directory-require.rkt is just meant to cure the tedium of importing the same file into every Pollen source file in your project. In a small project, not much tedium; in a large project, more.

Second, notice from the .rkt suffix that directory-require.rkt is a source file containing Racket code, not Pollen code. This is the default because while Pollen is better for text-driven source files, Racket is better for code-driven source files. Still, the choice is yours: the name of this file can be changed by resetting the world:directory-require value.

Third, notice from the directory- prefix that directory-require.rkt is only used by Pollen source files in the same directory. So if your project has source files nested inside a subdirectory, you’ll need to explicitly create another directory-require.rkt there and share the functions & values as needed.

“Why not make this file visible throughout a project, rather than just a directory?” Good idea, but I couldn’t figure out how to do it without creating finicky new dependencies. If you have a better idea, I’m open to it.

Let’s see how this works in practice. In the same directory as article.html.pm, create a new directory-require.rkt file as follows:

"directory-require.rkt"
#lang racket
(define author "Trevor Goodchild")
(provide author)

Here we use the define function (which we’ve seen before) to set author equal to "Trevor Goodchild". Note the final step: consistent with standard Racket rules, we have to explicitly provide the new value so that other files can see it (unlike Python, things you define in Racket are private by default, not public).

Then update good old article.html.pm:

"article.html.pm"
#lang pollen
 
The author is ◊|author|.

Run this in DrRacket and you’ll get:

'(root "The author is " "Trevor Goodchild" ".")

Now, in the same dirctory, create a second Pollen source file:

"barticle.html.pm"
#lang pollen
 
The author is really ◊|author|?

Run this, and you’ll get:

'(root "The author is really " "Trevor Goodchild" "?")

That’s all there is to it. Everything provided by directory-require.rkt is automatically available within each Pollen source file.

You can include functions, including tag functions, the same way. For instance, add a function for em:

"directory-require.rkt"
#lang racket
(define author "Trevor Goodchild")
(define (em . parts) `(extra (big ,@parts)))
(provide author em)

Then use it in a source file:

"article.html.pm"
#lang pollen
 
The ◊em{author} is ◊em{◊|author|}.

With the expected results:

'(root "The " (extra (big "author")) " is " (extra (big "Trevor Goodchild")) ".")

7.7 Decoding markup via the root tag

As you’ve seen, the X-expression you get when you run a Pollen markup file always starts with a node called root. You can attach a tag function to root the same way as any other tag. For instance, you could do something simple, like change the name of the output X-expression:

"article.html.pm"
#lang pollen
 
◊(define (root . elements) `(content ,@elements))
 
The ◊tt{root} tag is now called ◊tt{content}.

Resulting in:

'(content "The " (tt "root") " tag is now called " (tt "content") ".")

But unlike other tags in your document, root contains the entire content of the document. So the function you attach to root can operate on everything.

For that reason, one of the most useful things you can do with a tag function attached to root is decoding the content of the page. Decoding refers to any post-processing of content that happens after the tags within the page have been evaluated.

Decoding is a good way to automatically accomplish:

  • Detection of linebreaks, paragraphs, and list items based on whitespace.

  • Hyphenation.

  • Typographic optimizations, like smart quotes, dashes, and ligatures.

  • Gathering data for indexing or cross-referencing.

  • Any document enhancements a) that can be handled programmatically and b) that you’d prefer not to hard-code within your source files.

As an example, let’s take one of my favorites — linebreak and paragraph detection. In XML authoring, you have to insert every <br /> and <p> tag by hand. This is profoundly dull, clutters up the source file, and makes editing a chore.

Instead, let’s make a decoder that allows us to denote a linebreak with a single newline in the source, and a paragraph break with a double newline. Here’s some sample content with single and double newlines:

"article.html.pm"
#lang pollen
 
The first line of the 'first' paragraph.
And a new line.
 
The second paragraph --- isn't it great.

But without a decoder, the newlines just get passed through:

'(root "The first line of the 'first' paragraph." "\n" "And a new line." "\n" "\n" "The second paragraph --- isn't it great.")

When this X-expression is converted to HTML, the newlines persist:

<root>The first line of the 'first' paragraph.\nAnd a new line.\n\nThe second paragraph --- isn't it great.</root>

But in HTML, raw newlines are displayed as a single space. So if you view this file in the project server, you’ll see:

The first line of the 'first' paragraph. And a new line. The second paragraph --- isn't it great.

Not what we want.

So we need to make a decoder. To do this, we use the decode-elements function, which provides hooks to selectively process certain categories of content within the document.

decode-elements is a convenience variant of decode, which takes a full X-expression as input. Under the hood, they work the same way, so use whichever you prefer.

Add a basic decode-elements to the source file like so:

"article.html.pm"
#lang pollen
 
◊(require pollen/decode txexpr)
◊(define (root . elements)
   (make-txexpr 'root null (decode-elements elements)))
 
The first line of the 'first' paragraph.
And a new line.
 
The second paragraph --- isn't it great.

The make-txexpr function is a utility from the txexpr package, which is installed with Pollen. It builds a new X-expression from a tag, attribute list, and list of elements. Here, we’ll keep the tag name root, leave the attributes as null, and append our decoded list of elements.

Racket jocks: you could also write this using quasiquote and unquote-splicing syntax as `(root ,@(decode-elements elements)). The txexpr package is just a more explicit way of accomplishing the task.

If you run this file, what changes? Right — nothing. That’s because by default, both decode-elements (and decode) will let the content pass through unaltered.

We change this by giving decode-elements the name of a processing function and attaching it to the type of content we want to process. In this case, we’re in luck — the decode module already contains a detect-paragraphs function (that also detects linebreaks). We add this function using the keyword argument #:txexpr-elements-proc, which is short for “the function used to process the elements of a tagged X-expression”:

"article.html.pm"
#lang pollen
 
◊(require pollen/decode txexpr)
◊(define (root . elements)
   (make-txexpr 'root null (decode-elements elements
     #:txexpr-elements-proc detect-paragraphs)))
 
The first line of the 'first' paragraph.
And a new line.
 
The second paragraph --- isn't it great.

Now, when we run the file, the X-expression has changed to include two p tags and a br tag:

'(root (p "The first line of the 'first' paragraph." (br) "And a new line.") (p "The second paragraph --- isn't it great."))

That means when we convert to HTML, we get the tags we need:

<root><p>The first line of the 'first' paragraph.<br />And a new line.</p><p>The second paragraph --- isn't it great.</p></root>

So when we view this in the project server, the linebreaks and paragraph breaks are displayed correctly:

The first line of the 'first' paragraph.
And a new line.

The second paragraph --- isn't it great.

Of course, in practice you wouldn’t put your decoding function in a single source file. You’d make it available to all your source files by putting it in directory-require.rkt. So let’s do that now:

"directory-require.rkt"
#lang racket
(require pollen/decode txexpr)
(define (root . elements)
   (make-txexpr 'root null (decode-elements elements
     #:txexpr-elements-proc detect-paragraphs)))
(provide root)

We’ll also restore the source of article.html.pm to its original, simplified state:

"article.html.pm"
#lang pollen
 
The first line of the 'first' paragraph.
And a new line.
 
The second paragraph --- isn't it great.

And the result in the project server will be the same:

The first line of the 'first' paragraph.
And a new line.

The second paragraph --- isn't it great.

But wait, those straight quotes look terrible. Also, three hyphens for an em dash? Barbaric.

Let’s upgrade our decoder to take of those. Once again, we’ll get lucky, because the decode module provides two functions for the job: smart-quotes and smart-dashes.

This time, however, we’re going to attach them to another part of decode-elements. Smart-quote and smart-dash conversion only needs to look at the strings within the X-expression. So instead of attaching these functions to the #:txexpr-elements-proc argument of decode-elements, we’ll attach them to #:string-proc, which lets us specify a function to apply to strings:

"directory-require.rkt"
#lang racket/base
(require pollen/decode txexpr)
(define (root . elements)
   (make-txexpr 'root null (decode-elements elements
     #:txexpr-elements-proc detect-paragraphs
     #:string-proc (compose smart-quotes smart-dashes))))
(provide root)

Because #:string-proc only accepts one function (not two), we need to use compose to combine smart-quotes and smart-dashes into one (compose will apply the last function, then the previous one, and so on to the left end of the list).

Now, if we run article.html.pm in DrRacket, we can see the effects of the new decoder functions. The quotes are curled, and the three hyphens become an em dash:

'(root (p "The first line of the ‘first’ paragraph." (br) "And a new line.") (p "The second paragraph—isn’t it great."))

And of course, this shows up in the project server too:

The first line of the ‘first’ paragraph.
And a new line.

The second paragraph—isn’t it great.

By the way, even though decoding via the root tag is the most likely usage scenario, you don’t have to do it that way. Decoding is just a special kind of tag function. So you can make a decoder that only affects a certain tag within the page. Or you can make multiple decoders for different tags. The advantage of using a decoder with root is that it can affect all the content, and since it’s attached to the root node, it will always be the last tag function that gets called.

7.8 Putting it all together

For this final example, we’ll combine what we’ve learned in the first three tutorials. Though this project is still simple, it summarizes all the major concepts of Pollen.

It also provides a recipe you can adapt for your own projects, whether small or large. For instance, Butterick’s Practical Typography follows this core structure.

As we go through the ingredients, I’ll review the purpose of each. Save these files into a single project directory with the project server running.

7.8.1 The directory-require.rkt file

This file provides functions that are available to all Pollen source files in the same directory. It’s written in standard Racket. The directory-require.rkt file is optional — without it, your tags will just be treated as default tag functions. But you’ll probably find it a convenient way to make tag functions available within your project, including a decode function attached to root.

Here, we’ll use the directory-require.rkt we devised in the previous section to set up decoding for our source files:

"directory-require.rkt"
#lang racket/base
(require pollen/decode txexpr)
(define (root . elements)
   (make-txexpr 'root null (decode-elements elements
     #:txexpr-elements-proc detect-paragraphs
     #:string-proc (compose smart-quotes smart-dashes))))
(provide root)
7.8.2 The template

When you’re using Pollen authoring mode for your content — using either Markdown syntax, or Pollen markup — your source files will produce an X-expression. To convert this X-expression into a finished file, you need to use a template.

By default, when Pollen finds a source file called filename.ext.pm or filename.ext.pmd, it will look for a template in your project directory called template.ext, where .ext is the matching output extension.

In this project, we want to end up with HTML, so our source files will be called filename.html.pm, and thus we need to make a template.html. Let’s use a modified version of the one we made in the second tutorial:

"template.html"
<html>
<head>
<meta charset="UTF-8">
<title>◊select['h1 doc] by T. S. Eliot</title>
<link rel="stylesheet" type="text/css" media="all" href="styles.css" />
</head>
<body>◊->html[doc]
◊(define prev-page (previous here))
◊when/block[prev-page]{
<div id="prev">← <a href="◊|prev-page|">◊(select 'h1 prev-page)</a></div>}
◊(define next-page (next here))
◊when/block[next-page]{
<div id="next"><a href="◊|next-page|">◊(select 'h1 next-page)</a> →</div>}
</body>
</html>
7.8.3 The pagetree

A pagetree defines sequential and hierarchical relationships among a set of output files. The pagetree is used by the template to calculate navigational links (e.g., previous, next, up, etc.) A pagetree is optional — if you don’t need navigation in your project, you don’t need a pagetree.

But in this project, we do want navigation. So we’ll add an index.ptree file like so:

"index.ptree"
#lang pollen
 
burial.html
chess.html
sermon.html
7.8.4 A CSS stylesheet using the preprocessor

Our template file above refers to a CSS file called styles.css. When resolving linked files, the project server makes no distinction between static and dynamic files. If there’s a static file called styles.css, it will use that.

Or, if you make a preprocessor source file called styles.css.pp, it will be dynamically rendered into the requested styles.css file. The preprocessor will operate on any file with the .pp extension — so a preprocessor source called filename.ext.pp will be rendered into filename.ext. (The corollary is that preprocessor functionality can be added to any kind of text-based file.)

Preprocessor source files, like authoring source files, get access to everything in directory-require.rkt, so you can share common functions and variables.

Let’s use an improved version of the dynamic CSS file we made in the first tutorial.

"styles.css.pp"
#lang pollen
 
◊(define inner 2)
◊(define edge (* inner 2))
◊(define color "gray")
◊(define multiplier 1.3)
 
body {
    margin: ◊|edge|em;
    border: ◊|inner|em double ◊|color|;
    padding: ◊|inner|em;
    font-size: ◊|multiplier|em;
    line-height: ◊|multiplier|;
}
 
h1 {
    font-size: ◊|multiplier|em;
}
 
#prev, #next {
    position: fixed;
    top: ◊|(/ edge 2)|em;
}
 
#prev {
    left: ◊|edge|em;
}
 
#next {
    right: ◊|edge|em;
}
7.8.5 The content source files using Pollen markup

With the scaffolding in place, we need the content. Our pagetree contains three output files — burial.html, chess.html, and sermon.html. We’re going to make these output files using Pollen markup. So we’ll create three source files and name them by adding the .pm source extension to each of the output names — thus burial.html.pm, chess.html.pm, and sermon.html.pm, as follows (and with apologies to T. S. Eliot):

"burial.html.pm"
#lang pollen
 
◊h1{I. The Burial of the Dead}
 
"You gave me hyacinths first a year ago;
They called me the hyacinth girl."
--- Yet when we came back, late, from the Hyacinth garden,
Your arms full, and your hair wet, I could not
Speak, and my eyes failed, I was neither
Living nor dead, and I knew nothing,
Looking into the heart of light, the silence.
◊em{Od' und leer das Meer.}
 
Madame Sosostris, famous clairvoyante,
Had a bad cold, nevertheless
Is known to be the wisest woman in Europe,
With a wicked pack of cards.
"chess.html.pm"
#lang pollen
 
◊h1{II. A Game of Chess}
 
And still she cried, and still the world pursues,
"Jug Jug" to dirty ears.
And other withered stumps of time
Were told upon the walls; staring forms
Leaned out, leaning, hushing the room enclosed.
Footsteps shuffled on the stair,
Under the firelight, under the brush, her hair
Spread out in fiery points
Glowed into words, then would be savagely still.
 
"My nerves are bad to-night. Yes, bad. Stay with me.
Speak to me. Why do you never speak? Speak.
What are you thinking of? What thinking? What?
I never know what you are thinking. Think."
"sermon.html.pm"
#lang pollen
 
◊h1{III. The Fire Sermon}
 
"Trams and dusty trees.
Highbury bore me. Richmond and Kew
Undid me. By Richmond I raised my knees
Supine on the floor of a narrow canoe."
 
"My feet are at Moorgate, and my heart
Under my feet. After the event
He wept. He promised 'a new start.'
I made no comment. What should I resent?"
7.8.6 The result

Now visit the project server and view burial.html, which should look something like this (the box will expand to fit your browser window):

Click the navigational links at the top to move between pages. You’re encouraged to change the source files, the style sheet, the template, or directory-require.rkt, and see how these changes immediately affect the page rendering in the project server. (You can also change the sequence of the pages in index.ptree, but in that case, you’ll need to restart the project server to see the change.)

This page isn’t a miracle of web design, but it shows you in one example:

  • Pollen markup being decoded — paragraph breaks, linebreaks, smart quotes, smart dashes — with a decode function attached to the root node by directory-require.rkt;

  • A dynamically-generated CSS file that computes positions for CSS elements using numerical values set up with define, and mathematical conversions thereof;

  • Navigational links that appear and disappear as needed using conditional statements (when/block) in template.html, with the page sequence defined by index.ptree and the names of the links being pulled from the h1 tag of each source file using select.

7.9 Third tutorial complete

OK, that was a humongous tutorial. Congratulations on making it through.

But your reward is that you now understand all the core concepts of the Pollen publishing system, including the most important ones: the flexibility of Pollen markup, and the connection between tags and functions.

Armed with this knowledge, you have everything you need to start doing useful things with Pollen. I hope you enjoy using it as much as I’ve enjoyed making it!

    ← prev  up  next → 
 
\ No newline at end of file