diff --git a/doc/Decode.html b/doc/Decode.html index f3bce2d..4c8b407 100644 --- a/doc/Decode.html +++ b/doc/Decode.html @@ -1,2 +1,2 @@ -11.2 Decode
►Pollen:   the book is a program
▼11 Module reference
11.1 Cache
11.2 Decode
11.3 File
11.4 Pagetree
11.5 Render
11.6 Template
11.7 Tag
11.8 Top
11.9 World
►11.2 Decode
On this page:
decode
11.2.1 Block
project-block-tags
register-block-tag
block-txexpr?
11.2.2 Typography
whitespace?
whitespace/  nbsp?
smart-quotes
smart-dashes
detect-linebreaks
detect-paragraphs
wrap-hanging-quotes
6.1.0.5
    ← prev  up  next → 

11.2 Decode

 (require pollen/decode) package: pollen

The doc export of a Pollen markup file is a simple X-expression. Decoding refers to any post-processing of this X-expression. The pollen/decode module provides tools for creating decoders.

The decode step can happen separately from the compilation of the file. But you can also attach a decoder to the markup file’s root node, so the decoding happens automatically when the markup is compiled, and thus automatically incorporated into doc. (Following this approach, you could also attach multiple decoders to different tags within doc.)

You can, of course, embed function calls within Pollen markup. But since markup is optimized for authors, decoding is useful for operations that can or should be moved out of the authoring layer.

One example is presentation and layout. For instance, detect-paragraphs is a decoder function that lets authors mark paragraphs in their source simply by using two carriage returns.

Another example is conversion of output into a particular data format. Most Pollen functions are optimized for HTML output, but one could write a decoder that targets another format.

procedure

(decode tagged-xexpr    
  [#:txexpr-tag-proc txexpr-tag-proc    
  #:txexpr-attrs-proc txexpr-attrs-proc    
  #:txexpr-elements-proc txexpr-elements-proc    
  #:block-txexpr-proc block-txexpr-proc    
  #:inline-txexpr-proc inline-txexpr-proc    
  #:string-proc string-proc    
  #:symbol-proc symbol-proc    
  #:valid-char-proc valid-char-proc    
  #:cdata-proc cdata-proc    
  #:exclude-tags tags-to-exclude]) → txexpr?
  tagged-xexpr : txexpr?
  txexpr-tag-proc : (txexpr-tag? . -> . txexpr-tag?)
   = (λ(tag) tag)
  txexpr-attrs-proc : (txexpr-attrs? . -> . txexpr-attrs?)
   = (λ(attrs) attrs)
  txexpr-elements-proc : (txexpr-elements? . -> . txexpr-elements?)
   = (λ(elements) elements)
  block-txexpr-proc : (block-txexpr? . -> . xexpr?) = (λ(tx) tx)
  inline-txexpr-proc : (txexpr? . -> . xexpr?) = (λ(tx) tx)
  string-proc : (string? . -> . xexpr?) = (λ(str) str)
  symbol-proc : (symbol? . -> . xexpr?) = (λ(sym) sym)
  valid-char-proc : (valid-char? . -> . xexpr?) = (λ(vc) vc)
  cdata-proc : (cdata? . -> . xexpr?) = (λ(cdata) cdata)
  tags-to-exclude : (listof symbol?) = null
Recursively process a tagged-xexpr, usually the one exported from a Pollen source file as doc.

This function doesn’t do much on its own. Rather, it provides the hooks upon which harder-working functions can be hung.

Recall from [future link: Pollen mechanics] that any tag can have a function attached to it. By default, the tagged-xexpr from a source file is tagged with root. So the typical way to use decode is to attach your decoding functions to it, and then define root to invoke your decode function. Then it will be automatically applied to every doc during compile.

For instance, here’s how decode is attached to root in Butterick’s Practical Typography. There’s not much to it —

(define (root . items)
  (decode (make-txexpr 'root '() items)
          #:txexpr-elements-proc detect-paragraphs
          #:block-txexpr-proc (compose1 hyphenate wrap-hanging-quotes)
          #:string-proc (compose1 smart-quotes smart-dashes)
          #:exclude-tags '(style script)))

The hyphenate function is not part of Pollen, but rather the hyphenate package, which you can install separately.

This illustrates another important point: even though decode presents an imposing list of arguments, you’re unlikely to use all of them at once. These represent possibilities, not requirements. For instance, let’s see what happens when decode is invoked without any of its optional arguments.

Examples:

> (define tx '(root "I wonder" (em "why") "this works."))
> (decode tx)

'(root "I wonder" (em "why") "this works.")

Right — nothing. That’s because the default value for the decoding arguments is the identity function, (λ (x) x). So all the input gets passed through intact unless another action is specified.

The *-proc arguments of decode take procedures that are applied to specific categories of elements within txexpr.

The txexpr-tag-proc argument is a procedure that handles X-expression tags.

Examples:

> (define tx '(p "I'm from a strange" (strong "namespace")))
; Tags are symbols, so a tag-proc should return a symbol
> (decode tx #:txexpr-tag-proc (λ(t) (string->symbol (format "ns:~a" t))))

'(ns:p "I'm from a strange" (ns:strong "namespace"))

The txexpr-attrs-proc argument is a procedure that handles lists of X-expression attributes. (The txexpr module, included at no extra charge with Pollen, includes useful helper functions for dealing with these attribute lists.)

Examples:

> (define tx '(p [[id "first"]] "If I only had a brain."))
; Attrs is a list, so cons is OK for simple cases
> (decode tx #:txexpr-attrs-proc (λ(attrs) (cons '[class "PhD"] attrs)))

'(p ((class "PhD") (id "first")) "If I only had a brain.")

Note that txexpr-attrs-proc will change the attributes of every tagged X-expression, even those that don’t have attributes. This is useful, because sometimes you want to add attributes where none existed before. But be careful, because the behavior may make your processing function overinclusive.

Examples:

> (define tx '(div (p [[id "first"]] "If I only had a brain.")
  (p "Me too.")))
; This will insert the new attribute everywhere
> (decode tx #:txexpr-attrs-proc (λ(attrs) (cons '[class "PhD"] attrs)))

'(div

  ((class "PhD"))

  (p ((class "PhD") (id "first")) "If I only had a brain.")

  (p ((class "PhD")) "Me too."))

; This will add the new attribute only to non-null attribute lists
> (decode tx #:txexpr-attrs-proc
  (λ(attrs) (if (null? attrs) attrs (cons '[class "PhD"] attrs))))

'(div (p ((class "PhD") (id "first")) "If I only had a brain.") (p "Me too."))

The txexpr-elements-proc argument is a procedure that operates on the list of elements that represents the content of each tagged X-expression. Note that each element of an X-expression is subject to two passes through the decoder: once now, as a member of the list of elements, and also later, through its type-specific decoder (i.e., string-proc, symbol-proc, and so on).

Examples:

> (define tx '(div "Double" "\n" "toil" amp "trouble"))
; Every element gets doubled ...
> (decode tx #:txexpr-elements-proc (λ(es) (append-map (λ(e) `(,e ,e)) es)))

'(div "Double" "Double" "\n" "\n" "toil" "toil" amp amp "trouble" "trouble")

; ... but only strings get capitalized
> (decode tx #:txexpr-elements-proc (λ(es) (append-map (λ(e) `(,e ,e)) es))
  #:string-proc (λ(s) (string-upcase s)))

'(div "DOUBLE" "DOUBLE" "\n" "\n" "TOIL" "TOIL" amp amp "TROUBLE" "TROUBLE")

So why do you need txexpr-elements-proc? Because some types of element decoding depend on context, thus it’s necessary to handle the elements as a group. For instance, the doubling function above, though useless, requires handling the element list as a whole, because elements are being added.

A more useful example: paragraph detection. The behavior is not merely a map across each element:

Examples:

> (define (paras tx) (decode tx #:txexpr-elements-proc detect-paragraphs))
; Context matters. Trailing whitespace is ignored ...
> (paras '(body "The first paragraph." "\n\n"))

'(body "The first paragraph.")

; ... but whitespace between strings is converted to a break.
> (paras '(body "The first paragraph." "\n\n" "And another."))

'(body (p "The first paragraph.") (p "And another."))

; A combination of both types
> (paras '(body "The first paragraph." "\n\n" "And another." "\n\n"))

'(body (p "The first paragraph.") (p "And another."))

The block-txexpr-proc argument and the inline-txexpr-proc arguments are procedures that operate on tagged X-expressions. If the X-expression meets the block-txexpr? test, it is processed by block-txexpr-proc. Otherwise, it is processed by inline-txexpr-proc. Thus every tagged X-expression will be handled by one or the other. Of course, if you want block and inline elements to be handled the same way, you can set block-txexpr-proc and inline-txexpr-proc to be the same procedure.

Examples:

> (define tx '(div "Please" (em "mind the gap") (h1 "Tuesdays only")))
> (define add-ns (λ(tx) (make-txexpr
      (string->symbol (format "ns:~a" (get-tag tx)))
      (get-attrs tx)
      (get-elements tx))))
; div and h1 are block elements, so this will only affect them
> (decode tx #:block-txexpr-proc add-ns)

'(ns:div "Please" (em "mind the gap") (ns:h1 "Tuesdays only"))

; em is an inline element, so this will only affect it
> (decode tx #:inline-txexpr-proc add-ns)

'(div "Please" (ns:em "mind the gap") (h1 "Tuesdays only"))

; this will affect all elements
> (decode tx #:block-txexpr-proc add-ns #:inline-txexpr-proc add-ns)

'(ns:div "Please" (ns:em "mind the gap") (ns:h1 "Tuesdays only"))

The string-proc, symbol-proc, valid-char-proc, and cdata-proc arguments are procedures that operate on X-expressions that are strings, symbols, valid-chars, and CDATA, respectively. Deliberately, the output contracts for these procedures accept any kind of X-expression (meaning, the procedure can change the X-expression type).

Examples:

; A div with string, entity, character, and cdata elements
> (define tx `(div "Moe" amp 62 ,(cdata #f #f "3 > 2;")))
> (define rulify (λ(x) '(hr)))
; The rulify function is selectively applied to each
> (print (decode tx #:string-proc rulify))

'(div (hr) amp 62 #(struct:cdata #f #f "3 > 2;"))

> (print (decode tx #:symbol-proc rulify))

'(div "Moe" (hr) 62 #(struct:cdata #f #f "3 > 2;"))

> (print (decode tx #:valid-char-proc rulify))

'(div "Moe" amp (hr) #(struct:cdata #f #f "3 > 2;"))

> (print (decode tx #:cdata-proc rulify))

'(div "Moe" amp 62 (hr))

Finally, the tags-to-exclude argument is a list of tags that will be exempted from decoding. Though you could get the same result by testing the input within the individual decoding functions, that’s tedious and potentially slower.

Examples:

> (define tx '(p "I really think" (em "italics") "should be lowercase."))
> (decode tx #:string-proc (λ(s) (string-upcase s)))

'(p "I REALLY THINK" (em "ITALICS") "SHOULD BE LOWERCASE.")

> (decode tx #:string-proc (λ(s) (string-upcase s)) #:exclude-tags '(em))

'(p "I REALLY THINK" (em "italics") "SHOULD BE LOWERCASE.")

The tags-to-exclude argument is useful if you’re decoding source that’s destined to become HTML. According to the HTML spec, material within a <style> or <script> block needs to be preserved literally. In this example, if the CSS and JavaScript blocks are capitalized, they won’t work. So exclude '(style script), and problem solved.

Examples:

> (define tx '(body (h1 [[class "Red"]] "Let's visit Planet Telex.")
  (style [[type "text/css"]] ".Red {color: green;}")
  (script [[type "text/javascript"]] "var area = h * w;")))
> (decode tx #:string-proc (λ(s) (string-upcase s)))

'(body

  (h1 ((class "Red")) "LET'S VISIT PLANET TELEX.")

  (style ((type "text/css")) ".RED {COLOR: GREEN;}")

  (script ((type "text/javascript")) "VAR AREA = H * W;"))

> (decode tx #:string-proc (λ(s) (string-upcase s))
  #:exclude-tags '(style script))

'(body

  (h1 ((class "Red")) "LET'S VISIT PLANET TELEX.")

  (style ((type "text/css")) ".Red {color: green;}")

  (script ((type "text/javascript")) "var area = h * w;"))

11.2.1 Block

Because it’s convenient, Pollen categorizes tagged X-expressions into two categories: block and inline. Why is it convenient? When using decode, you often want to treat the two categories differently. Not that you have to. But this is how you can.

parameter

(project-block-tags) → (listof txexpr-tag?)

(project-block-tags block-tags) → void?
  block-tags : (listof txexpr-tag?)
A parameter that defines the set of tags that decode will treat as blocks. This parameter is initialized with the HTML block tags, namely:

(address article aside audio blockquote body canvas dd div dl fieldset figcaption figure footer form h1 h2 h3 h4 h5 h6 header hgroup noscript ol output p pre section table tfoot ul video)

procedure

(register-block-tag tag) → void?

  tag : txexpr-tag?
Adds a tag to project-block-tags so that block-txexpr? will report it as a block, and decode will process it with block-txexpr-proc rather than inline-txexpr-proc.

Pollen tries to do the right thing without being told. But this is the rare case where you have to be explicit. If you introduce a tag into your markup that you want treated as a block, you must use this function to identify it, or you will get spooky behavior later on.

For instance, detect-paragraphs knows that block elements in the markup shouldn’t be wrapped in a p tag. So if you introduce a new block element called bloq without registering it as a block, misbehavior will follow:

Examples:

> (define (paras tx) (decode tx #:txexpr-elements-proc detect-paragraphs))
> (paras '(body "I want to be a paragraph." "\n\n" (bloq "But not me.")))

'(body (p "I want to be a paragraph.") (p (bloq "But not me.")))

; Wrong: bloq should not be wrapped

But once you register bloq as a block, order is restored:

Examples:

> (define (paras tx) (decode tx #:txexpr-elements-proc detect-paragraphs))
> (register-block-tag 'bloq)
> (paras '(body "I want to be a paragraph." "\n\n" (bloq "But not me.")))

'(body (p "I want to be a paragraph.") (bloq "But not me."))

; Right: bloq is treated as a block

If you find the idea of registering block tags unbearable, good news. The project-block-tags include the standard HTML block tags by default. So if you just want to use things like div and p and h1–h6, you’ll get the right behavior for free.

Examples:

> (define (paras tx) (decode tx #:txexpr-elements-proc detect-paragraphs))
> (paras '(body "I want to be a paragraph." "\n\n" (div "But not me.")))

'(body (p "I want to be a paragraph.") (div "But not me."))

procedure

(block-txexpr? v) → boolean?

  v : any/c
Predicate that tests whether v is a tagged X-expression, and if so, whether the tag is among the project-block-tags. If not, it is treated as inline. To adjust how this test works, use register-block-tag.

11.2.2 Typography

An assortment of typography & layout functions, designed to be used with decode. These aren’t hard to write. So if you like these, use them. If not, make your own.

procedure

(whitespace? v) → boolean?

  v : any/c
A predicate that returns #t for any stringlike v that’s entirely whitespace, but also the empty string, as well as lists and vectors that are made only of whitespace? members. Following the regexp-match convention, whitespace? does not return #t for a nonbreaking space. If you prefer that behavior, use whitespace/nbsp?.

Examples:

> (whitespace? "\n\n   ")

#t

> (whitespace? (string->symbol "\n\n   "))

#t

> (whitespace? "")

#t

> (whitespace? '("" "  " "\n\n\n" " \n"))

#t

> (define nonbreaking-space (format "~a" #\u00A0))
> (whitespace? nonbreaking-space)

#f

procedure

(whitespace/nbsp? v) → boolean?

  v : any/c
Like whitespace?, but also returns #t for nonbreaking spaces.

Examples:

> (whitespace/nbsp? "\n\n   ")

#t

> (whitespace/nbsp? (string->symbol "\n\n   "))

#t

> (whitespace/nbsp? "")

#t

> (whitespace/nbsp? '("" "  " "\n\n\n" " \n"))

#t

> (define nonbreaking-space (format "~a" #\u00A0))
> (whitespace/nbsp? nonbreaking-space)

#t

procedure

(smart-quotes str) → string?

  str : string?
Convert straight quotes in str to curly according to American English conventions.

Examples:

> (define tricky-string
  "\"Why,\" she could've asked, \"are we in O‘ahu watching 'Mame'?\"")
> (display tricky-string)

"Why," she could've asked, "are we in O‘ahu watching 'Mame'?"

> (display (smart-quotes tricky-string))

“Why,” she could’ve asked, “are we in O‘ahu watching ‘Mame’?”

procedure

(smart-dashes str) → string?

  str : string?
In str, convert three hyphens to an em dash, and two hyphens to an en dash, and remove surrounding spaces.

Examples:

> (define tricky-string "I had a few --- OK, like 6--8 --- thin mints.")
> (display tricky-string)

I had a few --- OK, like 6--8 --- thin mints.

> (display (smart-dashes tricky-string))

I had a few—OK, like 6–8—thin mints.

; Monospaced font not great for showing dashes, but you get the idea

procedure

(detect-linebreaks tagged-xexpr-elements 
  [#:separator linebreak-sep 
  #:insert linebreak]) 
 → txexpr-elements?
  tagged-xexpr-elements : txexpr-elements?
  linebreak-sep : string? = world:linebreak-separator
  linebreak : xexpr? = '(br)
Within tagged-xexpr-elements, convert occurrences of linebreak-sep ("\n" by default) to linebreak, but only if linebreak-sep does not occur between blocks (see block-txexpr?). Why? Because block-level elements automatically display on a new line, so adding linebreak would be superfluous. In that case, linebreak-sep just disappears.

Examples:

> (detect-linebreaks '(div "Two items:" "\n" (em "Eggs") "\n" (em "Bacon")))

'(div "Two items:" (br) (em "Eggs") (br) (em "Bacon"))

> (detect-linebreaks '(div "Two items:" "\n" (div "Eggs") "\n" (div "Bacon")))

'(div "Two items:" (div "Eggs") (div "Bacon"))

procedure

(detect-paragraphs elements 
  [#:separator paragraph-sep 
  #:tag paragraph-tag 
  #:linebreak-proc linebreak-proc]) 
 → txexpr-elements?
  elements : txexpr-elements?
  paragraph-sep : string? = world:paragraph-separator
  paragraph-tag : symbol? = 'p
  linebreak-proc : (txexpr-elements? . -> . txexpr-elements?)
   = detect-linebreaks
Find paragraphs within elements (as denoted by paragraph-sep) and wrap them with paragraph-tag. Also handle linebreaks using detect-linebreaks.

If element is already a block-txexpr?, it will not be wrapped as a paragraph (because in that case, the wrapping would be superfluous). Thus, as a consequence, if paragraph-sep occurs between two blocks, it will be ignored (as in the example below using two sequential 'div blocks.)

The paragraph-tag argument sets the tag used to wrap paragraphs.

The linebreak-proc argument allows you to use a different linebreaking procedure other than the usual detect-linebreaks.

Examples:

> (detect-paragraphs '("First para" "\n\n" "Second para"))

'((p "First para") (p "Second para"))

> (detect-paragraphs '("First para" "\n\n" "Second para" "\n" "Second line"))

'((p "First para") (p "Second para" (br) "Second line"))

> (detect-paragraphs '("First para" "\n\n" (div "Second block")))

'((p "First para") (div "Second block"))

> (detect-paragraphs '((div "First block") "\n\n" (div "Second block")))

'((div "First block") (div "Second block"))

> (detect-paragraphs '("First para" "\n\n" "Second para") #:tag 'ns:p)

'((ns:p "First para") (ns:p "Second para"))

> (detect-paragraphs '("First para" "\n\n" "Second para" "\n" "Second line")
  #:linebreak-proc (λ(x) (detect-linebreaks x #:insert '(newline))))

'((p "First para") (p "Second para" (newline) "Second line"))

procedure

(wrap-hanging-quotes tx 
  [#:single-preprend single-preprender 
  #:double-preprend double-preprender]) 
 → txexpr?
  tx : txexpr?
  single-preprender : txexpr-tag? = 'squo
  double-preprender : txexpr-tag? = 'dquo
Find single or double quote marks at the beginning of tx and wrap them in an X-expression with the tag single-preprender or double-preprender, respectively. The default values are 'squo and 'dquo.

Examples:

> (wrap-hanging-quotes '(p "No quote to hang."))

'(p "No quote to hang.")

> (wrap-hanging-quotes '(p "“What? We need to hang quotes?”"))

'(p (dquo "“" "What? We need to hang quotes?”"))

In pro typography, quotation marks at the beginning of a line or paragraph are often shifted into the margin slightly to make them appear more optically aligned with the left edge of the text. With a reflowable layout model like HTML, you don’t know where your line breaks will be.

This function will simply insert the 'squo and 'dquo tags, which provide hooks that let you do the actual hanging via CSS, like so (actual measurement can be refined to taste):

squo {margin-left: -0.25em;}

dquo {margin-left: -0.50em;}

Be warned: there are many edge cases this function does not handle well.

Examples:

; Argh: this edge case is not handled properly
> (wrap-hanging-quotes '(p "“" (em "What?") "We need to hang quotes?”"))

'(p "“" (em "What?") "We need to hang quotes?”")

    ← prev  up  next → 
 
\ No newline at end of file +11.2 Decode
►Pollen:   the book is a program
▼11 Module reference
11.1 Cache
11.2 Decode
11.3 File
11.4 Pagetree
11.5 Render
11.6 Template
11.7 Tag
11.8 Top
11.9 World
►11.2 Decode
On this page:
decode
decode-elements
11.2.1 Block
project-block-tags
register-block-tag
block-txexpr?
11.2.2 Typography
whitespace?
whitespace/  nbsp?
smart-quotes
smart-dashes
detect-linebreaks
detect-paragraphs
wrap-hanging-quotes
6.1.0.5
    ← prev  up  next → 

11.2 Decode

 (require pollen/decode) package: pollen

The doc export of a Pollen markup file is a simple X-expression. Decoding refers to any post-processing of this X-expression. The pollen/decode module provides tools for creating decoders.

The decode step can happen separately from the compilation of the file. But you can also attach a decoder to the markup file’s root node, so the decoding happens automatically when the markup is compiled, and thus automatically incorporated into doc. (Following this approach, you could also attach multiple decoders to different tags within doc.)

You can, of course, embed function calls within Pollen markup. But since markup is optimized for authors, decoding is useful for operations that can or should be moved out of the authoring layer.

One example is presentation and layout. For instance, detect-paragraphs is a decoder function that lets authors mark paragraphs in their source simply by using two carriage returns.

Another example is conversion of output into a particular data format. Most Pollen functions are optimized for HTML output, but one could write a decoder that targets another format.

procedure

(decode tagged-xexpr    
  [#:txexpr-tag-proc txexpr-tag-proc    
  #:txexpr-attrs-proc txexpr-attrs-proc    
  #:txexpr-elements-proc txexpr-elements-proc    
  #:block-txexpr-proc block-txexpr-proc    
  #:inline-txexpr-proc inline-txexpr-proc    
  #:string-proc string-proc    
  #:symbol-proc symbol-proc    
  #:valid-char-proc valid-char-proc    
  #:cdata-proc cdata-proc    
  #:exclude-tags tags-to-exclude]) → txexpr?
  tagged-xexpr : txexpr?
  txexpr-tag-proc : (txexpr-tag? . -> . txexpr-tag?)
   = (λ(tag) tag)
  txexpr-attrs-proc : (txexpr-attrs? . -> . txexpr-attrs?)
   = (λ(attrs) attrs)
  txexpr-elements-proc : (txexpr-elements? . -> . txexpr-elements?)
   = (λ(elements) elements)
  block-txexpr-proc : (block-txexpr? . -> . xexpr?) = (λ(tx) tx)
  inline-txexpr-proc : (txexpr? . -> . xexpr?) = (λ(tx) tx)
  string-proc : (string? . -> . xexpr?) = (λ(str) str)
  symbol-proc : (symbol? . -> . xexpr?) = (λ(sym) sym)
  valid-char-proc : (valid-char? . -> . xexpr?) = (λ(vc) vc)
  cdata-proc : (cdata? . -> . xexpr?) = (λ(cdata) cdata)
  tags-to-exclude : (listof symbol?) = null
Recursively process a tagged-xexpr, usually the one exported from a Pollen source file as doc.

This function doesn’t do much on its own. Rather, it provides the hooks upon which harder-working functions can be hung.

Recall from [future link: Pollen mechanics] that any tag can have a function attached to it. By default, the tagged-xexpr from a source file is tagged with root. So the typical way to use decode is to attach your decoding functions to it, and then define root to invoke your decode function. Then it will be automatically applied to every doc during compile.

For instance, here’s how decode is attached to root in Butterick’s Practical Typography. There’s not much to it —

(define (root . items)
  (decode (make-txexpr 'root '() items)
          #:txexpr-elements-proc detect-paragraphs
          #:block-txexpr-proc (compose1 hyphenate wrap-hanging-quotes)
          #:string-proc (compose1 smart-quotes smart-dashes)
          #:exclude-tags '(style script)))

The hyphenate function is not part of Pollen, but rather the hyphenate package, which you can install separately.

This illustrates another important point: even though decode presents an imposing list of arguments, you’re unlikely to use all of them at once. These represent possibilities, not requirements. For instance, let’s see what happens when decode is invoked without any of its optional arguments.

Examples:

> (define tx '(root "I wonder" (em "why") "this works."))
> (decode tx)

'(root "I wonder" (em "why") "this works.")

Right — nothing. That’s because the default value for the decoding arguments is the identity function, (λ (x) x). So all the input gets passed through intact unless another action is specified.

The *-proc arguments of decode take procedures that are applied to specific categories of elements within txexpr.

The txexpr-tag-proc argument is a procedure that handles X-expression tags.

Examples:

> (define tx '(p "I'm from a strange" (strong "namespace")))
; Tags are symbols, so a tag-proc should return a symbol
> (decode tx #:txexpr-tag-proc (λ(t) (string->symbol (format "ns:~a" t))))

'(ns:p "I'm from a strange" (ns:strong "namespace"))

The txexpr-attrs-proc argument is a procedure that handles lists of X-expression attributes. (The txexpr module, included at no extra charge with Pollen, includes useful helper functions for dealing with these attribute lists.)

Examples:

> (define tx '(p [[id "first"]] "If I only had a brain."))
; Attrs is a list, so cons is OK for simple cases
> (decode tx #:txexpr-attrs-proc (λ(attrs) (cons '[class "PhD"] attrs)))

'(p ((class "PhD") (id "first")) "If I only had a brain.")

Note that txexpr-attrs-proc will change the attributes of every tagged X-expression, even those that don’t have attributes. This is useful, because sometimes you want to add attributes where none existed before. But be careful, because the behavior may make your processing function overinclusive.

Examples:

> (define tx '(div (p [[id "first"]] "If I only had a brain.")
  (p "Me too.")))
; This will insert the new attribute everywhere
> (decode tx #:txexpr-attrs-proc (λ(attrs) (cons '[class "PhD"] attrs)))

'(div

  ((class "PhD"))

  (p ((class "PhD") (id "first")) "If I only had a brain.")

  (p ((class "PhD")) "Me too."))

; This will add the new attribute only to non-null attribute lists
> (decode tx #:txexpr-attrs-proc
  (λ(attrs) (if (null? attrs) attrs (cons '[class "PhD"] attrs))))

'(div (p ((class "PhD") (id "first")) "If I only had a brain.") (p "Me too."))

The txexpr-elements-proc argument is a procedure that operates on the list of elements that represents the content of each tagged X-expression. Note that each element of an X-expression is subject to two passes through the decoder: once now, as a member of the list of elements, and also later, through its type-specific decoder (i.e., string-proc, symbol-proc, and so on).

Examples:

> (define tx '(div "Double" "\n" "toil" amp "trouble"))
; Every element gets doubled ...
> (decode tx #:txexpr-elements-proc (λ(es) (append-map (λ(e) `(,e ,e)) es)))

'(div "Double" "Double" "\n" "\n" "toil" "toil" amp amp "trouble" "trouble")

; ... but only strings get capitalized
> (decode tx #:txexpr-elements-proc (λ(es) (append-map (λ(e) `(,e ,e)) es))
  #:string-proc (λ(s) (string-upcase s)))

'(div "DOUBLE" "DOUBLE" "\n" "\n" "TOIL" "TOIL" amp amp "TROUBLE" "TROUBLE")

So why do you need txexpr-elements-proc? Because some types of element decoding depend on context, thus it’s necessary to handle the elements as a group. For instance, the doubling function above, though useless, requires handling the element list as a whole, because elements are being added.

A more useful example: paragraph detection. The behavior is not merely a map across each element:

Examples:

> (define (paras tx) (decode tx #:txexpr-elements-proc detect-paragraphs))
; Context matters. Trailing whitespace is ignored ...
> (paras '(body "The first paragraph." "\n\n"))

'(body "The first paragraph.")

; ... but whitespace between strings is converted to a break.
> (paras '(body "The first paragraph." "\n\n" "And another."))

'(body (p "The first paragraph.") (p "And another."))

; A combination of both types
> (paras '(body "The first paragraph." "\n\n" "And another." "\n\n"))

'(body (p "The first paragraph.") (p "And another."))

The block-txexpr-proc argument and the inline-txexpr-proc arguments are procedures that operate on tagged X-expressions. If the X-expression meets the block-txexpr? test, it is processed by block-txexpr-proc. Otherwise, it is processed by inline-txexpr-proc. Thus every tagged X-expression will be handled by one or the other. Of course, if you want block and inline elements to be handled the same way, you can set block-txexpr-proc and inline-txexpr-proc to be the same procedure.

Examples:

> (define tx '(div "Please" (em "mind the gap") (h1 "Tuesdays only")))
> (define add-ns (λ(tx) (make-txexpr
      (string->symbol (format "ns:~a" (get-tag tx)))
      (get-attrs tx)
      (get-elements tx))))
; div and h1 are block elements, so this will only affect them
> (decode tx #:block-txexpr-proc add-ns)

'(ns:div "Please" (em "mind the gap") (ns:h1 "Tuesdays only"))

; em is an inline element, so this will only affect it
> (decode tx #:inline-txexpr-proc add-ns)

'(div "Please" (ns:em "mind the gap") (h1 "Tuesdays only"))

; this will affect all elements
> (decode tx #:block-txexpr-proc add-ns #:inline-txexpr-proc add-ns)

'(ns:div "Please" (ns:em "mind the gap") (ns:h1 "Tuesdays only"))

The string-proc, symbol-proc, valid-char-proc, and cdata-proc arguments are procedures that operate on X-expressions that are strings, symbols, valid-chars, and CDATA, respectively. Deliberately, the output contracts for these procedures accept any kind of X-expression (meaning, the procedure can change the X-expression type).

Examples:

; A div with string, entity, character, and cdata elements
> (define tx `(div "Moe" amp 62 ,(cdata #f #f "3 > 2;")))
> (define rulify (λ(x) '(hr)))
; The rulify function is selectively applied to each
> (print (decode tx #:string-proc rulify))

'(div (hr) amp 62 #(struct:cdata #f #f "3 > 2;"))

> (print (decode tx #:symbol-proc rulify))

'(div "Moe" (hr) 62 #(struct:cdata #f #f "3 > 2;"))

> (print (decode tx #:valid-char-proc rulify))

'(div "Moe" amp (hr) #(struct:cdata #f #f "3 > 2;"))

> (print (decode tx #:cdata-proc rulify))

'(div "Moe" amp 62 (hr))

Finally, the tags-to-exclude argument is a list of tags that will be exempted from decoding. Though you could get the same result by testing the input within the individual decoding functions, that’s tedious and potentially slower.

Examples:

> (define tx '(p "I really think" (em "italics") "should be lowercase."))
> (decode tx #:string-proc (λ(s) (string-upcase s)))

'(p "I REALLY THINK" (em "ITALICS") "SHOULD BE LOWERCASE.")

> (decode tx #:string-proc (λ(s) (string-upcase s)) #:exclude-tags '(em))

'(p "I REALLY THINK" (em "italics") "SHOULD BE LOWERCASE.")

The tags-to-exclude argument is useful if you’re decoding source that’s destined to become HTML. According to the HTML spec, material within a <style> or <script> block needs to be preserved literally. In this example, if the CSS and JavaScript blocks are capitalized, they won’t work. So exclude '(style script), and problem solved.

Examples:

> (define tx '(body (h1 [[class "Red"]] "Let's visit Planet Telex.")
  (style [[type "text/css"]] ".Red {color: green;}")
  (script [[type "text/javascript"]] "var area = h * w;")))
> (decode tx #:string-proc (λ(s) (string-upcase s)))

'(body

  (h1 ((class "Red")) "LET'S VISIT PLANET TELEX.")

  (style ((type "text/css")) ".RED {COLOR: GREEN;}")

  (script ((type "text/javascript")) "VAR AREA = H * W;"))

> (decode tx #:string-proc (λ(s) (string-upcase s))
  #:exclude-tags '(style script))

'(body

  (h1 ((class "Red")) "LET'S VISIT PLANET TELEX.")

  (style ((type "text/css")) ".Red {color: green;}")

  (script ((type "text/javascript")) "var area = h * w;"))

procedure

(decode-elements elements 
  [#:txexpr-tag-proc txexpr-tag-proc 
  #:txexpr-attrs-proc txexpr-attrs-proc 
  #:txexpr-elements-proc txexpr-elements-proc 
  #:block-txexpr-proc block-txexpr-proc 
  #:inline-txexpr-proc inline-txexpr-proc 
  #:string-proc string-proc 
  #:symbol-proc symbol-proc 
  #:valid-char-proc valid-char-proc 
  #:cdata-proc cdata-proc 
  #:exclude-tags tags-to-exclude]) 
 → txexpr-elements?
  elements : txexpr-elements?
  txexpr-tag-proc : (txexpr-tag? . -> . txexpr-tag?)
   = (λ(tag) tag)
  txexpr-attrs-proc : (txexpr-attrs? . -> . txexpr-attrs?)
   = (λ(attrs) attrs)
  txexpr-elements-proc : (txexpr-elements? . -> . txexpr-elements?)
   = (λ(elements) elements)
  block-txexpr-proc : (block-txexpr? . -> . xexpr?) = (λ(tx) tx)
  inline-txexpr-proc : (txexpr? . -> . xexpr?) = (λ(tx) tx)
  string-proc : (string? . -> . xexpr?) = (λ(str) str)
  symbol-proc : (symbol? . -> . xexpr?) = (λ(sym) sym)
  valid-char-proc : (valid-char? . -> . xexpr?) = (λ(vc) vc)
  cdata-proc : (cdata? . -> . xexpr?) = (λ(cdata) cdata)
  tags-to-exclude : (listof symbol?) = null
Identical to decode, but takes txexpr-elements? as input rather than a whole tagged X-expression, and likewise returns txexpr-elements? rather than a tagged X-expression. A convenience variant for use inside tag functions.

11.2.1 Block

Because it’s convenient, Pollen puts tagged X-expressions into two categories: block and inline. Why is it convenient? When using decode, you often want to treat the two categories differently. Not that you have to. But this is how you can.

parameter

(project-block-tags) → (listof txexpr-tag?)

(project-block-tags block-tags) → void?
  block-tags : (listof txexpr-tag?)
A parameter that defines the set of tags that decode will treat as blocks. This parameter is initialized with the HTML block tags, namely:

(address article aside audio blockquote body canvas dd div dl fieldset figcaption figure footer form h1 h2 h3 h4 h5 h6 header hgroup noscript ol output p pre section table tfoot ul video)

procedure

(register-block-tag tag) → void?

  tag : txexpr-tag?
Adds a tag to project-block-tags so that block-txexpr? will report it as a block, and decode will process it with block-txexpr-proc rather than inline-txexpr-proc.

Pollen tries to do the right thing without being told. But this is the rare case where you have to be explicit. If you introduce a tag into your markup that you want treated as a block, you must use this function to identify it, or you will get spooky behavior later on.

For instance, detect-paragraphs knows that block elements in the markup shouldn’t be wrapped in a p tag. So if you introduce a new block element called bloq without registering it as a block, misbehavior will follow:

Examples:

> (define (paras tx) (decode tx #:txexpr-elements-proc detect-paragraphs))
> (paras '(body "I want to be a paragraph." "\n\n" (bloq "But not me.")))

'(body (p "I want to be a paragraph.") (p (bloq "But not me.")))

; Wrong: bloq should not be wrapped

But once you register bloq as a block, order is restored:

Examples:

> (define (paras tx) (decode tx #:txexpr-elements-proc detect-paragraphs))
> (register-block-tag 'bloq)
> (paras '(body "I want to be a paragraph." "\n\n" (bloq "But not me.")))

'(body (p "I want to be a paragraph.") (bloq "But not me."))

; Right: bloq is treated as a block

If you find the idea of registering block tags unbearable, good news. The project-block-tags include the standard HTML block tags by default. So if you just want to use things like div and p and h1–h6, you’ll get the right behavior for free.

Examples:

> (define (paras tx) (decode tx #:txexpr-elements-proc detect-paragraphs))
> (paras '(body "I want to be a paragraph." "\n\n" (div "But not me.")))

'(body (p "I want to be a paragraph.") (div "But not me."))

procedure

(block-txexpr? v) → boolean?

  v : any/c
Predicate that tests whether v is a tagged X-expression, and if so, whether the tag is among the project-block-tags. If not, it is treated as inline. To adjust how this test works, use register-block-tag.

11.2.2 Typography

An assortment of typography & layout functions, designed to be used with decode. These aren’t hard to write. So if you like these, use them. If not, make your own.

procedure

(whitespace? v) → boolean?

  v : any/c
A predicate that returns #t for any stringlike v that’s entirely whitespace, but also the empty string, as well as lists and vectors that are made only of whitespace? members. Following the regexp-match convention, whitespace? does not return #t for a nonbreaking space. If you prefer that behavior, use whitespace/nbsp?.

Examples:

> (whitespace? "\n\n   ")

#t

> (whitespace? (string->symbol "\n\n   "))

#t

> (whitespace? "")

#t

> (whitespace? '("" "  " "\n\n\n" " \n"))

#t

> (define nonbreaking-space (format "~a" #\u00A0))
> (whitespace? nonbreaking-space)

#f

procedure

(whitespace/nbsp? v) → boolean?

  v : any/c
Like whitespace?, but also returns #t for nonbreaking spaces.

Examples:

> (whitespace/nbsp? "\n\n   ")

#t

> (whitespace/nbsp? (string->symbol "\n\n   "))

#t

> (whitespace/nbsp? "")

#t

> (whitespace/nbsp? '("" "  " "\n\n\n" " \n"))

#t

> (define nonbreaking-space (format "~a" #\u00A0))
> (whitespace/nbsp? nonbreaking-space)

#t

procedure

(smart-quotes str) → string?

  str : string?
Convert straight quotes in str to curly according to American English conventions.

Examples:

> (define tricky-string
  "\"Why,\" she could've asked, \"are we in O‘ahu watching 'Mame'?\"")
> (display tricky-string)

"Why," she could've asked, "are we in O‘ahu watching 'Mame'?"

> (display (smart-quotes tricky-string))

“Why,” she could’ve asked, “are we in O‘ahu watching ‘Mame’?”

procedure

(smart-dashes str) → string?

  str : string?
In str, convert three hyphens to an em dash, and two hyphens to an en dash, and remove surrounding spaces.

Examples:

> (define tricky-string "I had a few --- OK, like 6--8 --- thin mints.")
> (display tricky-string)

I had a few --- OK, like 6--8 --- thin mints.

> (display (smart-dashes tricky-string))

I had a few—OK, like 6–8—thin mints.

; Monospaced font not great for showing dashes, but you get the idea

procedure

(detect-linebreaks tagged-xexpr-elements 
  [#:separator linebreak-sep 
  #:insert linebreak]) 
 → txexpr-elements?
  tagged-xexpr-elements : txexpr-elements?
  linebreak-sep : string? = world:linebreak-separator
  linebreak : xexpr? = '(br)
Within tagged-xexpr-elements, convert occurrences of linebreak-sep ("\n" by default) to linebreak, but only if linebreak-sep does not occur between blocks (see block-txexpr?). Why? Because block-level elements automatically display on a new line, so adding linebreak would be superfluous. In that case, linebreak-sep just disappears.

Examples:

> (detect-linebreaks '(div "Two items:" "\n" (em "Eggs") "\n" (em "Bacon")))

'(div "Two items:" (br) (em "Eggs") (br) (em "Bacon"))

> (detect-linebreaks '(div "Two items:" "\n" (div "Eggs") "\n" (div "Bacon")))

'(div "Two items:" (div "Eggs") (div "Bacon"))

procedure

(detect-paragraphs elements 
  [#:separator paragraph-sep 
  #:tag paragraph-tag 
  #:linebreak-proc linebreak-proc]) 
 → txexpr-elements?
  elements : txexpr-elements?
  paragraph-sep : string? = world:paragraph-separator
  paragraph-tag : symbol? = 'p
  linebreak-proc : (txexpr-elements? . -> . txexpr-elements?)
   = detect-linebreaks
Find paragraphs within elements (as denoted by paragraph-sep) and wrap them with paragraph-tag. Also handle linebreaks using detect-linebreaks.

If element is already a block-txexpr?, it will not be wrapped as a paragraph (because in that case, the wrapping would be superfluous). Thus, as a consequence, if paragraph-sep occurs between two blocks, it will be ignored (as in the example below using two sequential 'div blocks.)

The paragraph-tag argument sets the tag used to wrap paragraphs.

The linebreak-proc argument allows you to use a different linebreaking procedure other than the usual detect-linebreaks.

Examples:

> (detect-paragraphs '("First para" "\n\n" "Second para"))

'((p "First para") (p "Second para"))

> (detect-paragraphs '("First para" "\n\n" "Second para" "\n" "Second line"))

'((p "First para") (p "Second para" (br) "Second line"))

> (detect-paragraphs '("First para" "\n\n" (div "Second block")))

'((p "First para") (div "Second block"))

> (detect-paragraphs '((div "First block") "\n\n" (div "Second block")))

'((div "First block") (div "Second block"))

> (detect-paragraphs '("First para" "\n\n" "Second para") #:tag 'ns:p)

'((ns:p "First para") (ns:p "Second para"))

> (detect-paragraphs '("First para" "\n\n" "Second para" "\n" "Second line")
  #:linebreak-proc (λ(x) (detect-linebreaks x #:insert '(newline))))

'((p "First para") (p "Second para" (newline) "Second line"))

procedure

(wrap-hanging-quotes tx 
  [#:single-preprend single-preprender 
  #:double-preprend double-preprender]) 
 → txexpr?
  tx : txexpr?
  single-preprender : txexpr-tag? = 'squo
  double-preprender : txexpr-tag? = 'dquo
Find single or double quote marks at the beginning of tx and wrap them in an X-expression with the tag single-preprender or double-preprender, respectively. The default values are 'squo and 'dquo.

Examples:

> (wrap-hanging-quotes '(p "No quote to hang."))

'(p "No quote to hang.")

> (wrap-hanging-quotes '(p "“What? We need to hang quotes?”"))

'(p (dquo "“" "What? We need to hang quotes?”"))

In pro typography, quotation marks at the beginning of a line or paragraph are often shifted into the margin slightly to make them appear more optically aligned with the left edge of the text. With a reflowable layout model like HTML, you don’t know where your line breaks will be.

This function will simply insert the 'squo and 'dquo tags, which provide hooks that let you do the actual hanging via CSS, like so (actual measurement can be refined to taste):

squo {margin-left: -0.25em;}

dquo {margin-left: -0.50em;}

Be warned: there are many edge cases this function does not handle well.

Examples:

; Argh: this edge case is not handled properly
> (wrap-hanging-quotes '(p "“" (em "What?") "We need to hang quotes?”"))

'(p "“" (em "What?") "We need to hang quotes?”")

    ← prev  up  next → 
 
\ No newline at end of file diff --git a/doc/doc-index.html b/doc/doc-index.html index 0a52200..dd76b4b 100644 --- a/doc/doc-index.html +++ b/doc/doc-index.html @@ -1,2 +1,2 @@ -Index
▼Pollen:   the book is a program
    ← prev  up  next → 

Index

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

 

“But I really need XML…”
“Now you have two problems”
#%top
->html
->markup-source-path
->null-source-path
->output-path
->pagenode
->preproc-source-path
->scribble-source-path
->template-source-path
A special data structure for HTML
Acknowledgments
Adding commands
Adding navigation links to the template with here
Any command is valid
Attaching behavior to tags
Attributes
Authoring mode
Backstory
Block
block-txexpr?
Cache
cache-ref
cached-require
children
Choosing custom tags
Command syntax using ◊
Creating a Pollen markup file
Creating a source file
Creating valid HTML output
current-cache
current-pagetree
Custom exports
Decode
decode
def/c
Defining variables with commands
detect-linebreaks
detect-paragraphs
Development environment
Enter Racket
File
File formats
First tutorial
First tutorial complete
Format independence
Functions
Further reading
get-template-for
Handling navigation boundaries with conditionals
has-markup-source?
has-null-source?
has-preproc-source?
has-scribble-source?
has-template-source?
has/is-markup-source?
has/is-null-source?
has/is-preproc-source?
has/is-scribble-source?
has/is-template-source?
in-pagetree?
index.ptree & the project server
Inserting a comment
Inserting metas
Inserting specific source data into templates
Inserting the value of a variable
Inserting values from variables
Inserting variables within CSS
Installation
Intermission
Intermission
Intermission
Interpolating variables into strings
Invoking other functions
Invoking tag functions
License & source code
Linking to an external CSS file
make-cache
make-default-tag-function
Making a custom template
Making a pagetree file
Making pagetrees by hand
Making pagetrees with a source file
Making sure raco pollen works
Markdown (.pmd extension)
Markdown authoring mode
Markdown in Pollen: two options
Markdown mode
Markup (.pm extension)
Markup mode
markup-source?
Module reference
Multiple input values & rest arguments
Naming, saving, and rendering a source file
Navigation
next
next*
Notes for experienced programmers
Null (.p extension)
null-source?
One language, multiple dialects
Organizing functions
pagenode?
pagenodeish?
Pagetree
Pagetree (.ptree extension)
Pagetree navigation
pagetree->list
pagetree-source?
pagetree?
Pagetrees
Pagetrees
parent
Parsing attributes
path->pagenode
Point of no return
pollen
Pollen as a preprocessor
Pollen command syntax
Pollen markup vs. XML
pollen/cache
pollen/decode
pollen/file
pollen/markdown
pollen/markup
pollen/pagetree
pollen/pre
pollen/ptree
pollen/render
pollen/tag
pollen/template
pollen/top
pollen/world
Pollen: the book is a program
Predicates & validation
Prelude: my principled objection to Markdown
preproc-source?
Preprocessor (.pp extension)
Prerequisites
Prerequisites
Prerequisites
previous
previous*
project-block-tags
PS for Scribble users
Putting in the text of the poem
Putting it all together
Quick tour
Racket basics (if you’re not familiar)
raco pollen
raco pollen clone
raco pollen help
raco pollen render
raco pollen start
register-block-tag
Render
render
render-batch
render-pagetree
render-to-file
render-to-file-if-needed
reset-cache
Rethinking the solution for digital books
Returning an X-expression
Running a source file
Saving & naming your source file
Scribble (.scrbl extension)
scribble-source?
Second tutorial
Second tutorial complete
select
select*
select-from-doc
select-from-metas
Semantic markup
Setting the #lang line
Setting up a preprocessor source file
siblings
smart-dashes
smart-quotes
Source files in the dashboard
Source formats
split-attributes
Standard exports
Starting a new file in DrRacket
Starting the project server with raco pollen
Tag
Tags & tag functions
Tags are functions
Template
template-source?
Templated source files
Templates
Templates
The ->html function and the doc variable
The better idea: a programming model
The big picture
The book is a program
The command name
The directory-require.rkt file
The directory-require.rkt file
The end of the beginning
The golden rule
The lozenge glyph (◊)
The preprocessor
The project server
The Racket arguments
The relationship of Racket & Pollen
The text argument
The two command modes: text mode & Racket mode
The XML problem
Third tutorial
Top
Typography
Using custom tags
Using index.ptree in the dashboard
Using Markdown with the preprocessor
Using pagetrees for navigation
Using pagetrees with raco pollen render
Using Racket’s function libraries
Using raco pollen
Using the automatic pagetree
Using the dashboard
Using the project server
Utilities
Utility formats
validate-pagetree
Web development and its discontents
What are custom tags good for?
What is Pollen?
What Pollen markup does differently
when/block
whitespace/nbsp?
whitespace?
Working with the preprocessor
World
world:check-directory-requires-in-render?
world:command-marker
world:current-server-extras-path
world:current-server-port
world:dashboard-css
world:decodable-extensions
world:default-pagetree
world:default-port
world:default-template-prefix
world:directory-require
world:fallback-template-prefix
world:linebreak-separator
world:main-pollen-export
world:markdown-source-ext
world:markup-source-ext
world:meta-pollen-export
world:mode-auto
world:mode-markdown
world:mode-markup
world:mode-pagetree
world:mode-preproc
world:newline
world:null-source-ext
world:pagetree-root-node
world:pagetree-source-ext
world:paragraph-separator
world:paths-excluded-from-dashboard
world:preproc-source-ext
world:scribble-source-ext
world:server-extras-dir
world:template-meta-key
world:template-source-ext
wrap-hanging-quotes
Writing with Pollen markup
X-expressions
◊ command overview

    ← prev  up  next → 
 
\ No newline at end of file +Index
▼Pollen:   the book is a program
    ← prev  up  next → 

Index

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

 

“But I really need XML…”
“Now you have two problems”
#%top
->html
->markup-source-path
->null-source-path
->output-path
->pagenode
->preproc-source-path
->scribble-source-path
->template-source-path
A special data structure for HTML
Acknowledgments
Adding commands
Adding navigation links to the template with here
Any command is valid
Attaching behavior to tags
Attributes
Authoring mode
Backstory
Block
block-txexpr?
Cache
cache-ref
cached-require
children
Choosing custom tags
Command syntax using ◊
Creating a Pollen markup file
Creating a source file
Creating valid HTML output
current-cache
current-pagetree
Custom exports
Decode
decode
decode-elements
Decoding markup via the root tag
def/c
Defining variables with commands
detect-linebreaks
detect-paragraphs
Development environment
Enter Racket
File
File formats
First tutorial
First tutorial complete
Format independence
Functions
Further reading
get-template-for
Handling navigation boundaries with conditionals
has-markup-source?
has-null-source?
has-preproc-source?
has-scribble-source?
has-template-source?
has/is-markup-source?
has/is-null-source?
has/is-preproc-source?
has/is-scribble-source?
has/is-template-source?
in-pagetree?
index.ptree & the project server
Inserting a comment
Inserting metas
Inserting specific source data into templates
Inserting the value of a variable
Inserting values from variables
Inserting variables within CSS
Installation
Intermission
Intermission
Intermission
Interpolating variables into strings
Invoking other functions
Invoking tag functions
License & source code
Linking to an external CSS file
make-cache
make-default-tag-function
Making a custom template
Making a pagetree file
Making pagetrees by hand
Making pagetrees with a source file
Making sure raco pollen works
Markdown (.pmd extension)
Markdown authoring mode
Markdown in Pollen: two options
Markdown mode
Markup (.pm extension)
Markup mode
markup-source?
Module reference
Multiple input values & rest arguments
Naming, saving, and rendering a source file
Navigation
next
next*
Notes for experienced programmers
Null (.p extension)
null-source?
One language, multiple dialects
Organizing functions
pagenode?
pagenodeish?
Pagetree
Pagetree (.ptree extension)
Pagetree navigation
pagetree->list
pagetree-source?
pagetree?
Pagetrees
Pagetrees
parent
Parsing attributes
path->pagenode
Point of no return
pollen
Pollen as a preprocessor
Pollen command syntax
Pollen markup vs. XML
pollen/cache
pollen/decode
pollen/file
pollen/markdown
pollen/markup
pollen/pagetree
pollen/pre
pollen/ptree
pollen/render
pollen/tag
pollen/template
pollen/top
pollen/world
Pollen: the book is a program
Predicates & validation
Prelude: my principled objection to Markdown
preproc-source?
Preprocessor (.pp extension)
Prerequisites
Prerequisites
Prerequisites
previous
previous*
project-block-tags
PS for Scribble users
Putting in the text of the poem
Putting it all together
Quick tour
Racket basics (if you’re not familiar)
raco pollen
raco pollen clone
raco pollen help
raco pollen render
raco pollen start
register-block-tag
Render
render
render-batch
render-pagetree
render-to-file
render-to-file-if-needed
reset-cache
Rethinking the solution for digital books
Returning an X-expression
Running a source file
Saving & naming your source file
Scribble (.scrbl extension)
scribble-source?
Second tutorial
Second tutorial complete
select
select*
select-from-doc
select-from-metas
Semantic markup
Setting the #lang line
Setting up a preprocessor source file
siblings
smart-dashes
smart-quotes
Source files in the dashboard
Source formats
split-attributes
Standard exports
Starting a new file in DrRacket
Starting the project server with raco pollen
Tag
Tags & tag functions
Tags are functions
Template
template-source?
Templated source files
Templates
Templates
The ->html function and the doc variable
The better idea: a programming model
The big picture
The book is a program
The command name
The directory-require.rkt file
The directory-require.rkt file
The end of the beginning
The golden rule
The lozenge glyph (◊)
The preprocessor
The project server
The Racket arguments
The relationship of Racket & Pollen
The text argument
The two command modes: text mode & Racket mode
The XML problem
Third tutorial
Top
Typography
Using custom tags
Using index.ptree in the dashboard
Using Markdown with the preprocessor
Using pagetrees for navigation
Using pagetrees with raco pollen render
Using Racket’s function libraries
Using raco pollen
Using the automatic pagetree
Using the dashboard
Using the project server
Utilities
Utility formats
validate-pagetree
Web development and its discontents
What are custom tags good for?
What is Pollen?
What Pollen markup does differently
when/block
whitespace/nbsp?
whitespace?
Working with the preprocessor
World
world:check-directory-requires-in-render?
world:command-marker
world:current-server-extras-path
world:current-server-port
world:dashboard-css
world:decodable-extensions
world:default-pagetree
world:default-port
world:default-template-prefix
world:directory-require
world:fallback-template-prefix
world:linebreak-separator
world:main-pollen-export
world:markdown-source-ext
world:markup-source-ext
world:meta-pollen-export
world:mode-auto
world:mode-markdown
world:mode-markup
world:mode-pagetree
world:mode-preproc
world:newline
world:null-source-ext
world:pagetree-root-node
world:pagetree-source-ext
world:paragraph-separator
world:paths-excluded-from-dashboard
world:preproc-source-ext
world:scribble-source-ext
world:server-extras-dir
world:template-meta-key
world:template-source-ext
wrap-hanging-quotes
Writing with Pollen markup
X-expressions
◊ command overview

    ← prev  up  next → 
 
\ No newline at end of file diff --git a/doc/index.html b/doc/index.html index 32f79a5..e9af041 100644 --- a/doc/index.html +++ b/doc/index.html @@ -1,3 +1,3 @@ Pollen: the book is a program
▼Pollen:   the book is a program
On this page:
Pollen:   the book is a program
6.1.0.5
    ← prev  up  next → 

Pollen: the book is a program

Matthew Butterick <mb@mbtype.com>

 #lang pollen package: pollen

Pollen is a publishing system that helps authors create beautiful and functional web-based books. Pollen includes tools for writing, designing, programming, testing, and publishing.

I used Pollen to create my book Butterick’s Practical Typography. Sure, go take a look. Is it better than the last digital book you encountered? Yes it is. Would you like your book to look like that? If so, keep reading.

At the core of Pollen is an argument: -

  • First, that digital books should be the best books we’ve ever had. So far, they’re not even close.

  • Second, that because digital books are software, an author shouldn’t think of a book as merely data. The book is a program.

  • Third, that the way we make digital books better than their predecessors is by exploiting this programmability.

That’s what Pollen is for.

Not that you need to be a programmer to use Pollen. On the contrary, the Pollen language is markup-based, so you can write & edit text naturally. But when you want to automate repetitive tasks, add cross-references, or pull in data from other sources, you can access a full programming language from within the text.

That language is Racket. I chose Racket because while the idea for Pollen had been with me for several years, it simply wasn’t possible to build it with other languages. So if it’s unfamiliar to you, don’t panic. It was unfamiliar to me. Once you see what you can do with Pollen & Racket, you may be persuaded. I was.

Or, if you can find a better digital-publishing tool, use that. But I’m never going back to the way I used to work.

    1 Installation

    2 Quick tour

      2.1 Creating a source file

      2.2 Running a source file

      2.3 Naming, saving, and rendering a source file

      2.4 The project server

      2.5 Intermission

      2.6 Pollen as a preprocessor

      2.7 Markdown mode

      2.8 Markup mode

      2.9 Templates

      2.10 PS for Scribble users

      2.11 The end of the beginning

    3 Backstory

      3.1 Web development and its discontents

      3.2 The better idea: a programming model

      3.3 “Now you have two problems”

      3.4 Rethinking the solution for digital books

      3.5 Enter Racket

      3.6 What is Pollen?

    4 The big picture

      4.1 The book is a program

      4.2 One language, multiple dialects

      4.3 Development environment

      4.4 A special data structure for HTML

      4.5 Pollen command syntax

      4.6 The preprocessor

      4.7 Templated source files

      4.8 Pagetrees

    5 First tutorial

      5.1 Prerequisites

      5.2 The relationship of Racket & Pollen

      5.3 Starting a new file in DrRacket

        5.3.1 Setting the #lang line

        5.3.2 Putting in the text of the poem

        5.3.3 Saving & naming your source file

      5.4 Using the project server

        5.4.1 Starting the project server with raco pollen

        5.4.2 Using the dashboard

        5.4.3 Source files in the dashboard

      5.5 Working with the preprocessor

        5.5.1 Setting up a preprocessor source file

        5.5.2 Creating valid HTML output

        5.5.3 Adding commands

        5.5.4 Racket basics (if you’re not familiar)

        5.5.5 Defining variables with commands

        5.5.6 Inserting values from variables

        5.5.7 Inserting variables within CSS

      5.6 First tutorial complete

    6 Second tutorial

      6.1 Prerequisites

      6.2 Prelude: my principled objection to Markdown

      6.3 Markdown in Pollen: two options

        6.3.1 Using Markdown with the preprocessor

        6.3.2 Authoring mode

        6.3.3 X-expressions

        6.3.4 Markdown authoring mode

      6.4 Templates

        6.4.1 The ->html function and the doc variable

        6.4.2 Making a custom template

        6.4.3 Inserting specific source data into templates

        6.4.4 Linking to an external CSS file

      6.5 Intermission

      6.6 Pagetrees

        6.6.1 Pagetree navigation

        6.6.2 Using the automatic pagetree

        6.6.3 Adding navigation links to the template with here

        6.6.4 Handling navigation boundaries with conditionals

        6.6.5 Making a pagetree file

        6.6.6 index.ptree & the project server

      6.7 Second tutorial complete

    7 Third tutorial

      7.1 Prerequisites

      7.2 Pollen markup vs. XML

        7.2.1 The XML problem

        7.2.2 What Pollen markup does differently

        7.2.3 “But I really need XML…”

      7.3 Writing with Pollen markup

        7.3.1 Creating a Pollen markup file

        7.3.2 Tags & tag functions

        7.3.3 Attributes

        7.3.4 What are custom tags good for?

        7.3.5 Semantic markup

        7.3.6 Format independence

        7.3.7 Using custom tags

        7.3.8 Choosing custom tags

      7.4 Tags are functions

        7.4.1 Attaching behavior to tags

        7.4.2 Notes for experienced programmers

          7.4.2.1 Point of no return

          7.4.2.2 Multiple input values & rest arguments

          7.4.2.3 Returning an X-expression

          7.4.2.4 Interpolating variables into strings

          7.4.2.5 Parsing attributes

      7.5 Intermission

      7.6 Organizing functions

        7.6.1 Using Racket’s function libraries

        7.6.2 The directory-require.rkt file

      7.7 Putting it all together

    8 Using raco pollen

      8.1 Making sure raco pollen works

      8.2 raco pollen

      8.3 raco pollen help

      8.4 raco pollen start

      8.5 raco pollen render

      8.6 raco pollen clone

    9 File formats

      9.1 Source formats

        9.1.1 Command syntax using ◊

        9.1.2 Any command is valid

        9.1.3 Standard exports

        9.1.4 Custom exports

        9.1.5 The directory-require.rkt file

        9.1.6 Preprocessor (.pp extension)

        9.1.7 Markdown (.pmd extension)

        9.1.8 Markup (.pm extension)

        9.1.9 Pagetree (.ptree extension)

      9.2 Utility formats

        9.2.1 Scribble (.scrbl extension)

        9.2.2 Null (.p extension)

    10 ◊ command overview

      10.1 The golden rule

      10.2 The lozenge glyph (◊)

      10.3 The two command modes: text mode & Racket mode

        10.3.1 The command name

          10.3.1.1 Invoking tag functions

          10.3.1.2 Invoking other functions

          10.3.1.3 Inserting the value of a variable

          10.3.1.4 Inserting metas

          10.3.1.5 Inserting a comment

        10.3.2 The Racket arguments

        10.3.3 The text argument

      10.4 Further reading

    11 Module reference

      11.1 Cache

      11.2 Decode

        11.2.1 Block

        11.2.2 Typography

      11.3 File

      11.4 Pagetree

        11.4.1 Making pagetrees with a source file

        11.4.2 Making pagetrees by hand

        11.4.3 Using pagetrees for navigation

        11.4.4 Using index.ptree in the dashboard

        11.4.5 Using pagetrees with raco pollen render

        11.4.6 Functions

          11.4.6.1 Predicates & validation

          11.4.6.2 Navigation

          11.4.6.3 Utilities

      11.5 Render

      11.6 Template

      11.7 Tag

      11.8 Top

      11.9 World

    12 Acknowledgments

    13 License & source code

    Index

    ← prev  up  next → 
 
\ No newline at end of file +

That’s what Pollen is for.

Not that you need to be a programmer to use Pollen. On the contrary, the Pollen language is markup-based, so you can write & edit text naturally. But when you want to automate repetitive tasks, add cross-references, or pull in data from other sources, you can access a full programming language from within the text.

That language is Racket. I chose Racket because while the idea for Pollen had been with me for several years, it simply wasn’t possible to build it with other languages. So if it’s unfamiliar to you, don’t panic. It was unfamiliar to me. Once you see what you can do with Pollen & Racket, you may be persuaded. I was.

Or, if you can find a better digital-publishing tool, use that. But I’m never going back to the way I used to work.

    1 Installation

    2 Quick tour

      2.1 Creating a source file

      2.2 Running a source file

      2.3 Naming, saving, and rendering a source file

      2.4 The project server

      2.5 Intermission

      2.6 Pollen as a preprocessor

      2.7 Markdown mode

      2.8 Markup mode

      2.9 Templates

      2.10 PS for Scribble users

      2.11 The end of the beginning

    3 Backstory

      3.1 Web development and its discontents

      3.2 The better idea: a programming model

      3.3 “Now you have two problems”

      3.4 Rethinking the solution for digital books

      3.5 Enter Racket

      3.6 What is Pollen?

    4 The big picture

      4.1 The book is a program

      4.2 One language, multiple dialects

      4.3 Development environment

      4.4 A special data structure for HTML

      4.5 Pollen command syntax

      4.6 The preprocessor

      4.7 Templated source files

      4.8 Pagetrees

    5 First tutorial

      5.1 Prerequisites

      5.2 The relationship of Racket & Pollen

      5.3 Starting a new file in DrRacket

        5.3.1 Setting the #lang line

        5.3.2 Putting in the text of the poem

        5.3.3 Saving & naming your source file

      5.4 Using the project server

        5.4.1 Starting the project server with raco pollen

        5.4.2 Using the dashboard

        5.4.3 Source files in the dashboard

      5.5 Working with the preprocessor

        5.5.1 Setting up a preprocessor source file

        5.5.2 Creating valid HTML output

        5.5.3 Adding commands

        5.5.4 Racket basics (if you’re not familiar)

        5.5.5 Defining variables with commands

        5.5.6 Inserting values from variables

        5.5.7 Inserting variables within CSS

      5.6 First tutorial complete

    6 Second tutorial

      6.1 Prerequisites

      6.2 Prelude: my principled objection to Markdown

      6.3 Markdown in Pollen: two options

        6.3.1 Using Markdown with the preprocessor

        6.3.2 Authoring mode

        6.3.3 X-expressions

        6.3.4 Markdown authoring mode

      6.4 Templates

        6.4.1 The ->html function and the doc variable

        6.4.2 Making a custom template

        6.4.3 Inserting specific source data into templates

        6.4.4 Linking to an external CSS file

      6.5 Intermission

      6.6 Pagetrees

        6.6.1 Pagetree navigation

        6.6.2 Using the automatic pagetree

        6.6.3 Adding navigation links to the template with here

        6.6.4 Handling navigation boundaries with conditionals

        6.6.5 Making a pagetree file

        6.6.6 index.ptree & the project server

      6.7 Second tutorial complete

    7 Third tutorial

      7.1 Prerequisites

      7.2 Pollen markup vs. XML

        7.2.1 The XML problem

        7.2.2 What Pollen markup does differently

        7.2.3 “But I really need XML…”

      7.3 Writing with Pollen markup

        7.3.1 Creating a Pollen markup file

        7.3.2 Tags & tag functions

        7.3.3 Attributes

        7.3.4 What are custom tags good for?

        7.3.5 Semantic markup

        7.3.6 Format independence

        7.3.7 Using custom tags

        7.3.8 Choosing custom tags

      7.4 Tags are functions

        7.4.1 Attaching behavior to tags

        7.4.2 Notes for experienced programmers

          7.4.2.1 Point of no return

          7.4.2.2 Multiple input values & rest arguments

          7.4.2.3 Returning an X-expression

          7.4.2.4 Interpolating variables into strings

          7.4.2.5 Parsing attributes

      7.5 Intermission

      7.6 Organizing functions

        7.6.1 Using Racket’s function libraries

        7.6.2 The directory-require.rkt file

      7.7 Decoding markup via the root tag

      7.8 Putting it all together

    8 Using raco pollen

      8.1 Making sure raco pollen works

      8.2 raco pollen

      8.3 raco pollen help

      8.4 raco pollen start

      8.5 raco pollen render

      8.6 raco pollen clone

    9 File formats

      9.1 Source formats

        9.1.1 Command syntax using ◊

        9.1.2 Any command is valid

        9.1.3 Standard exports

        9.1.4 Custom exports

        9.1.5 The directory-require.rkt file

        9.1.6 Preprocessor (.pp extension)

        9.1.7 Markdown (.pmd extension)

        9.1.8 Markup (.pm extension)

        9.1.9 Pagetree (.ptree extension)

      9.2 Utility formats

        9.2.1 Scribble (.scrbl extension)

        9.2.2 Null (.p extension)

    10 ◊ command overview

      10.1 The golden rule

      10.2 The lozenge glyph (◊)

      10.3 The two command modes: text mode & Racket mode

        10.3.1 The command name

          10.3.1.1 Invoking tag functions

          10.3.1.2 Invoking other functions

          10.3.1.3 Inserting the value of a variable

          10.3.1.4 Inserting metas

          10.3.1.5 Inserting a comment

        10.3.2 The Racket arguments

        10.3.3 The text argument

      10.4 Further reading

    11 Module reference

      11.1 Cache

      11.2 Decode

        11.2.1 Block

        11.2.2 Typography

      11.3 File

      11.4 Pagetree

        11.4.1 Making pagetrees with a source file

        11.4.2 Making pagetrees by hand

        11.4.3 Using pagetrees for navigation

        11.4.4 Using index.ptree in the dashboard

        11.4.5 Using pagetrees with raco pollen render

        11.4.6 Functions

          11.4.6.1 Predicates & validation

          11.4.6.2 Navigation

          11.4.6.3 Utilities

      11.5 Render

      11.6 Template

      11.7 Tag

      11.8 Top

      11.9 World

    12 Acknowledgments

    13 License & source code

    Index

    ← prev  up  next → 
 
\ No newline at end of file diff --git a/doc/third-tutorial.html b/doc/third-tutorial.html index ba69716..6283c0e 100644 --- a/doc/third-tutorial.html +++ b/doc/third-tutorial.html @@ -1,3 +1,9 @@ -7 Third tutorial
▼Pollen:   the book is a program
►7 Third tutorial
On this page:
7.1 Prerequisites
7.2 Pollen markup vs. XML
7.2.1 The XML problem
7.2.2 What Pollen markup does differently
7.2.3 “But I really need XML…”
7.3 Writing with Pollen markup
7.3.1 Creating a Pollen markup file
7.3.2 Tags & tag functions
7.3.3 Attributes
7.3.4 What are custom tags good for?
7.3.5 Semantic markup
7.3.6 Format independence
7.3.7 Using custom tags
7.3.8 Choosing custom tags
7.4 Tags are functions
7.4.1 Attaching behavior to tags
7.4.2 Notes for experienced programmers
7.4.2.1 Point of no return
7.4.2.2 Multiple input values & rest arguments
7.4.2.3 Returning an X-expression
7.4.2.4 Interpolating variables into strings
7.4.2.5 Parsing attributes
7.5 Intermission
7.6 Organizing functions
7.6.1 Using Racket’s function libraries
7.6.2 The directory-require.rkt file
7.7 Putting it all together
6.1.0.5
    ← prev  up  next → 

7 Third tutorial

Now you’re getting to the good stuff. In this tutorial, you’ll use Pollen to publish a multi-page article written in Pollen markup. You’ll learn about:

If you want the shortest possible introduction to Pollen, try the Quick tour.

7.1 Prerequisites

I’ll assume you’ve completed the Second tutorial and that you understand the principles of Pollen authoring mode — creating source files, converting them to X-expressions, and then combining them with templates to make output files.

Because now it’s time to pick up the pace. You’ve learned how to do some handy things with Pollen. But we haven’t yet exploited the full fusion of writing environment and programming language. I promised you that The book is a program, right? So let’s do some programming.

7.2 Pollen markup vs. XML

You can skip this section if XML holds no interest. But Pollen markup evolved out of my attempt to come up with an alternative to XML that would be more usable for writing. So if you’re familiar with XML, the contrast may be helpful.

7.2.1 The XML problem

In the Second tutorial, I made the case that Markdown is a limiting format for authors. Why? Markdown is essentially a notation system for HTML tags. As such, it has three problems: it’s not semantic, it only covers a limited subset of HTML tags, and it can’t be extended by an author.

These problems are partly limitations of HTML itself. And these limitations were meant to be cured by XML — the X stands for extensible. In principle, XML allows you to define whatever tags you like and use them in your document.

So why hasn’t XML taken over the world? In practice, XML promises more than it delivers. The reasons are apparent to any writer who’s attempted to use XML as an authoring format:

The nicest thing we could say about XML is that its intentions are good. It’s oriented toward the right goals. But its benefits are buried under atrocious ergonomics.

7.2.2 What Pollen markup does differently

Pollen markup can be seen as a way of reaping the benefits of XML without incurring the headaches. Like XML, Pollen markup allows you to freely tag your text. But unlike XML:

7.2.3 “But I really need XML…”

You can have XML. There’s nothing wrong with using Pollen markup to generate XML files that can then be fed into an existing XML processing pipeline. In other words, using Pollen markup, you can treat XML as an output format rather than an input format.

In this tutorial, I’ll be rendering Pollen markup with an HTML template. But you could easily use the same workflow with an XML template and thus end up with XML files.

7.3 Writing with Pollen markup

Pollen markup is a free-form markup system that lets you add arbitrary tags and attributes to your text. By arbitrary, I mean that they don’t need to match up with an existing schema or specification (e.g., the tags permitted by HTML). They can — but that’s an option, not a requirement.

I like to think of Pollen markup a way of capturing not just the text, but also my ideas about the text. Some of these are low-level ideas (“this text should be italicized”). Some are high-level ideas (“this text is the topic of the page”). Some are just notes to myself. In short, everything I know about the text becomes part of the text.

In so doing, Pollen markup becomes the source code of the book. Let’s try it out.

7.3.1 Creating a Pollen markup file

We’re going to use Pollen markup to make a file that will ultimately be HTML. So consistent with the authoring-mode workflow we learned in the Second tutorial, we’ll start with our desired output filename, article.html, and then append the Pollen markup suffix, .pm.

In DrRacket, start a new file called article.html.pm like so (BTW you can use any sample text you like):

"article.html.pm"
#lang pollen
 
I want to attend RacketCon this year.

Consistent with usual authoring-mode policy, when you run this file, you’ll get an X-expression that starts with root:

'(root "I want to attend RacketCon this year.")

Remember, even though the first line of the file is #lang pollen — same as the last tutorial — the new .pm suffix signals that Pollen should interpret the source as Pollen markup. Look what happens if you goof up and put Markdown source in a Pollen markup file, like so:

#lang pollen
 
I am **so** excited to attend __RacketCon__ this year.

The Markdown syntax will be ignored, and pass through to the output:

'(root "I am **so** excited to attend __RacketCon__ this year.")

Restore the non-Markdown source, and let’s continue.

7.3.2 Tags & tag functions

Pollen markup uses the same Pollen command syntax that we first saw in Adding commands. Previously, we used this command syntax to invoke functions like define and ->html. Pollen markup is used to invoke a special kind of function called a tag function, which is a function that, by default, adds a tag to the text.

To see how this works, restore your article.html.pm file to its original state:

#lang pollen
 
I want to attend RacketCon this year.

We can add any tag with Pollen markup, but for now, let’s start with an old favorite: em, which is used in HTML to add emphasis to text. We apply a tag by starting with the lozenge character (◊) followed by the tag name em, followed by the text in curly braces, like so:

"article.html.pm"
#lang pollen
 
I want to attend ◊em{RacketCon this year}.

Run this file in DrRacket and see the X-expression that results:

'(root "I want to attend " (em "RacketCon this year") ".")

You won’t be surprised to hear that you can nest tags:

"article.html.pm"
#lang pollen
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

With the expected results:

'(root "I want to attend " (em "RacketCon " (strong "this") " year") ".")

7.3.3 Attributes

Attributes are like tags for tags. Each attribute is a key–value pair where the key is any name, and the value is a string. Anyone who’s seen HTML is familiar with them:

<span class="author">Prof. Leonard</span>

Here, class is an attribute for span that has value "author". And this is what it looks like as an X-expression:

'(span ((class "author")) "Prof. Leonard")

You can add any number of attributes to a tag (first as an X-expression, then as HTML):

'(span ((class "author")(id "primary")(living "true")) "Prof. Leonard")

<span class="author" id="primary" living="true">Prof. Leonard</span>

In Pollen markup, attributes have the same logic, but a slightly different syntax. In keeping with the tag notation you just saw, the span tag is added in the usual way:

"article.html.pm"
#lang pollen
 
◊span{Prof. Leonard}

Then you have two options for adding attributes. The verbose way corresponds to how the attributes appear in the X-expression:

"article.html.pm"
#lang pollen
 
◊span['((class "author")(id "primary")(living "true"))]{Prof. Leonard}

Each key–value pair is in parentheses, and then the list of pairs is within parentheses, with a quote (') at the front that signals that the text should be used literally.

This involves some superfluous typing, however, so Pollen also supports an abbreviated syntax for attributes:

"article.html.pm"
#lang pollen
 
◊span['class:"author" 'id:"primary" 'living:"true"]{Prof. Leonard}

In this form, each attribute key starts with a quote mark ' and ends with a colon :. As before, the attribute value is in quotation marks.

Both of these forms will produce the same X-expression:

'(span ((class "author")(id "primary")(living "true")) "Prof. Leonard")

Now that you know how to make tags and attributes, you might wonder whether Pollen markup can be used as a quick & dirty HTML-notation system. Sure — for a quick & dirty project, why not. Recall that X-expressions are just alternative notation for the standard angle-bracket notation used in HTML. So if you wanted HTML like this:

<div class="red" style="font-size:150%">Important <em>News</em></div>

You could write it in Pollen markup like so:

◊div['class:"red" style:"font-size:150%"]{Important ◊em{News}}

And then just convert it (using the ->html function) into the HTML above. Thus, the tags you already know and love (?) can be used in Pollen markup, but with fewer keystrokes and cruft.

Still, if Pollen markup were just an alternative notation system for HTML tags, it would be pretty boring. As I alluded above, that’s merely a boring way to use it.

In the XML spirit, Pollen markup lets you use any tags you want. That’s considerably less boring.

7.3.4 What are custom tags good for?

XML jocks can skip this section, since you already know. But if you’ve been mired in Markdown or HTML, read on.

Tags, broadly speaking, are a means of annotating a text with extra information, which I’ll call metadata (using that term in its generic sense, not in any fiddly computery way). Metadata is the key tool that enables an author to write a book with the benefits of semantic markup and format independence.

7.3.5 Semantic markup

Semantic markup means adding metadata to text according to the meaning of the text, not merely its intended visual appearance. So rather than tagging RacketCon with an em tag, as we did above to indicate how the word should look, maybe we would tag it with an event tag, to indicate what kind of thing it is.

Semantic markup lets an author specify distinctions that would be ambiguous in pure visual terms, thereby capturing more meaning and intent. For instance, in books, italic styling is commonly applied to a number of unrelated types of information: emphasized words, movie titles, terms being used for the first time, headings, captions and labels, and so on. Under a non-semantic formatting scheme, perhaps one would tag them all em. But in semantic terms, one would tag them movie-title, first-use, heading, as appropriate.

This has two major benefits. First, by separating appearance and meaning, an author can manage the content of the book in useful ways. For instance, if every movie title were tagged as movie-title rather than italic, then it would be simple to generate a list of all movies mentioned in the book (for the author’s benefit) or a page index of movie references (for the reader’s benefit). But without that semantic tagging, a movie title couldn’t be distinguished from any other italicized text.

7.3.6 Format independence

The second benefit of custom tags is format independence, or the ability to change the rendering of the text to suit a particular device or context.

When a text is encrusted with format-specific visual tags — for instance, HTML tags — then the document markup is entangled with a single output format. If you only need one output format, fine.

But increasingly, book authors have been called upon to publish their work in multiple formats: paper and PDF, but also web, e-book, or other natively digital formats, that connect to devices with differing display capabilities.

Yes, I know that many of these formats are based on variants of HTML. But the HTML you can use in a desktop web browser is quite different from, say, the HTML you can use in a Kindle .mobi file. The .mobi file has other technical requirements too, like an .ncx and .opf file. So despite some genetic kinship, these HTML-ish formats are best understood as separate targets.

Using a display-driven model to manage this complexity is a terrible idea — as anyone who’s tried it can attest. Converting from one display-based file type to another — for instance, word processor to HTML, or HTML to PDF — is an exercise in frustration and drain-circling expectations.

This isn’t surprising. For a long time, text processing has been dominated by this display-driven model. Most word processors, like Microsoft Word and Pages, have been built around this model. It worked well enough in the era where most documents were eventually going to be printed on paper (or a paper simulator like PDF). HTML was a technical leap forward, but not a conceptual leap: it mostly represented the display options available in a web browser.

There’s a couple TeX fans at the back of the room, waving their arms. Yes, TeX got a lot of things right. In practice, however, it never became a core tool for electronic publishing (which, to be fair, didn’t exist when TeX was written). Plenty of ideas in Pollen were lifted from TeX.

For a document to be format independent, two conditions have to be satisfied.

First, the document has to be readable by other programs, so they can handle the conversion of format-independent markup into a format-specific rendering (e.g., mapping semantic tags like movie-title onto visual tags like em). Most word-processor formats, like Word’s .docx, are bad for authoring because these formats are opaque and proprietary. We needn’t get into the political objections. As a practical matter, they’re inarguably restrictive — if you can’t get your data out of your file, you’re stuck.

Second, the document itself has to be represented in a way that’s independent of the particularities of any one format. For instance, HTML is a bad authoring format because it encourages authors to litter their text with HTML-isms like h1 and span. These have no meaning outside of HTML, and thus will always cause conversion problems. The same goes for Markdown, which is simply HTML in disguise.

The solution to the first condition is to use text-based markup rather than proprietary file types. The solution to the second condition is to let authors define custom tags for the document, rather than the other way around. Pollen markup incorporates both of these ideas.

7.3.7 Using custom tags

You can insert a custom tag using the same syntax as any other tag. Suppose you want to use an event tag to mark events. You would insert it like so:

"article.html.pm"
#lang pollen
 
I want to attend ◊event{RacketCon} this year.

This markup will turn into this X-expression:

'(root "I want to attend " (event "RacketCon") " this year.")

Which is equivalent to this XML:

<root>I want to attend <event>RacketCon</event> this year.</root>

In truth, Pollen doesn’t notice any difference between a custom tag vs. a standard HTML tag vs. any other kind of tag. They’re all just markup tags. If you want to restrict yourself to a certain vocabulary of tags, you can. If you want to set up Pollen to enforce those restrictions, you can do that too. But by default, Pollen doesn’t impose restrictions like this. In general, you can pick any tag name you want, and it will work.

Don’t take my word for it. See what happens if you write this:

"article.html.pm"
#lang pollen
 
I want to attend ◊verylongandimpracticaltagname{RacketCon} this year.

One small but important exception to this rule. If you were wondering why I sometimes call them tag functions instead of just tags, it’s because under the hood, every tag is implemented as a function. The default behavior of this function is just to wrap the text in a tag with the given name.

The benefit of treating tags as functions will become evident later in this tutorial. But the cost of this approach is that tags occupy the same namespace as the other functions available in Pollen (and by extension, Racket). So if you try to use a tag name that’s already the name of an existing function, an error will occur.

For instance, let’s suppose you try to use a custom tag called length:

"article.html.pm"
#lang pollen
 
The Panama Canal is ◊length{77km} across.

When you run this file, you get an error:

length: contract violation;
expected: list?
  given: "77km"

The problem is that Racket already provides a function called length. Consistent with the usual rules of Pollen command notation, your command is interpreted as an attempt to invoke the length function, rather than apply a tag named length.

In practice, namespace clashes are rare. But if necessary, they’re easy to work around (for the simplest method, see Invoking tag functions).

7.3.8 Choosing custom tags

You just saw that using custom tags is easy. Choosing custom tags, on the other hand, is less science than art. As the author, it’s up to you. Some guidelines:

And most important:

As we’ll see in the next section, this is where your book truly becomes programmable.

7.4 Tags are functions

Don’t skip this section! It explains a concept that’s essential to understanding how Pollen works.

If you’ve used HTML or XML, tags are just tags: things you type into the document that look the same going out as they did going in. Tags can be used to select document elements or assign styling (via CSS). But they don’t have any deeper effect on the document content.

That’s not so in Pollen. Under the hood, Pollen is just an alternate way of writing code in the Racket programming language. And tags, instead of being inert markers, are actually functions.

I think most of you know what a function is, but just to be safe — in programming, a function is a chunk of code that accepts some input, processes it, and then returns a value. Asking a function to process some data is known as calling the function.

Leading us to the Three Golden Rules of Pollen Tags:

  1. Every Pollen tag calls a function with the same name.

  2. The input values for that function are the attributes and content of the tag.

  3. The whole tag — tag name, attributes, and content — is replaced with the return value of the called function.

You’ve already seen the simplest kind of function in a Pollen document: the default tag function, which emulates the behavior of standard markup tags.

Let’s revisit an earlier example, now with the help of the Golden Rules:

"article.html.pm"
#lang pollen
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

What happens when you run this source? Working from the inside out, Pollen calls the function strong with the input "this". The result is (strong "this"). Then Pollen calls the function em with the three input values "RacketCon " (strong "this") " year", which yields (em "RacketCon " (strong "this") " year"). Finally, Pollen calls the root function with everything in the document, resulting in:

'(root "I want to attend " (em "RacketCon " (strong "this") " year") ".")

7.4.1 Attaching behavior to tags

Sometimes this default behavior will suffice. But other times, you’ll want to change the behavior of a tag. Why? Here are some useful examples of what you, as an author, can do with custom tag functions:

Having invited you to gaze across these vistas, I apologize that my example here in this tutorial is necessarily tip-of-the-iceberg. I’ll be adding a more detailed guide to writing Pollen functions, both simple and crafty.

How do you change the behavior of a tag? By 1) writing a new function and 2) giving it the name of the tag. Once you do this, this new behavior will automatically be invoked when you use the tag.

For example, let’s redefine the strong tag in our example above to simply print BOOM:

"article.html.pm"
#lang pollen
 
◊define[(strong . lines)]{BOOM}
 
I want to attend ◊em{RacketCon ◊strong{this} year}

When you run this file, you indeed get:

'(root "I want to attend " (em "RacketCon " "BOOM" " year"))

How does this work? First, although you can define a function in Pollen command syntax using either of The two command modes: text mode & Racket mode, it tends to be easier to use Racket mode. I wrote the first one in text mode. But for clarity, I’m going to switch to Racket mode (run this file and convince yourself it comes out the same):

"article.html.pm"
#lang pollen
 
◊(define (strong word) "BOOM")
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

Let’s look at our new function definition. As usual, we start with the lozenge character (◊) to denote a Pollen command. Then we use define to introduce a function definition. The name of the function comes next, which needs to match our tag name, strong. The expression (strong word) means “the name of this function is strong, and it takes a single word as input, which we’ll refer to as word.” Finally we have the return value, which is "BOOM".

Let’s run this file again, but go back to the Golden Rules to understand what happens. Working from the inside out, Pollen calls the function strong with the input "this" — same as before. But this time, the result of the strong function is not (strong "this"), but simply BOOM. Then Pollen calls the function em with the three input values "RacketCon " "BOOM" " year", which yields (em "RacketCon " "BOOM" " year"). Finally, Pollen calls the root function with everything in the document, resulting in:

'(root "I want to attend " (em "RacketCon " "BOOM" " year"))

This example is contrived, of course. But the basic idea — defining a function with the name of a tag — is the foundation of programmability in Pollen. If you get this, and the Golden Rules, you get everything.

7.4.2 Notes for experienced programmers

Having said that, some of you are probably eager to hack around a bit. Let me chip off a few more cubes from the iceberg to help you on your way. (Everyone else, take five.)

7.4.2.1 Point of no return

If you’ve written functions in other programming languages, you might be accustomed to using a return statement to send a value back from the function. This doesn’t exist in Pollen or Racket — the return value of any function is just the last expression evaluated. In the example below, "BAP" becomes the return value because it’s in the last position, and "BOOM" is ignored:

"article.html.pm"
#lang pollen
 
◊(define (strong word) "BOOM" "BAP")
 
I want to attend ◊em{RacketCon ◊strong{this} year}.
7.4.2.2 Multiple input values & rest arguments

Sometimes a tag will have only one word or string that becomes its input. More likely, however, it will have multiple values (this is inevitable with nested tags, because the results aren’t concatenated). For instance, if we attach our function to em rather than strong:

"article.html.pm"
#lang pollen
 
◊(define (em word) "BOOM")
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

Look what happens:

em: arity mismatch;
the expected number of arguments does not match the given number
expected: 1
  given: 3

The error arises because the em function is getting three arguments — "RacketCon " "BOOM" " year" — but has been defined to only accept one argument, word. This is the “arity mismatch.”

To fix this, it’s better to get in the habit of writing tag functions that accept an indefinite number of input values. You do this by defining your function with a rest argument (as in, “give me the rest of the input values.”) To use a rest argument, put it last in your list of input arguments, and add a period . before:

"article.html.pm"
#lang pollen
 
◊(define (em . parts) "BOOM")
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

This time, the source file will run without an error, producing this:

'(root "I want to attend " "BOOM" ".")

A rest argument like parts is a list of individual arguments. So if you want to unpack & process these arguments separately, you can use Racket’s extensive list-processing functions (see Pairs and Lists). Also see quasiquote below.

7.4.2.3 Returning an X-expression

Often, you won’t use a tag function to replace a whole tag with a string — you’ll replace it with a different tag, described by an X-expression, like so:

"article.html.pm"
#lang pollen
 
◊(define (em . parts) '(big "BOOM"))
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

Which produces:

'(root "I want to attend " (big "BOOM") ".")

The quote mark ' before the X-expression signals to Racket that you want to use what follows as a literal value.

To build X-expressions that are more elaborate, you have two options.

First is quasiquote. Quasiquote works like quote, but starts with a backtick character `. What makes it “quasi” is that you can insert variables using the unquote operator, which is a comma , or merge a list of values with the unquote-splicing operator, which is a comma followed by an @ sign ,@.

Let’s adapt the example above to use quasiquote. Suppose we want to take the parts we get as input and put them inside a big tag. This is easy to notate with quasiquote and the unquote-splicing operator, because parts is a list:

"article.html.pm"
#lang pollen
 
◊(define (em . parts) `(big ,@parts))
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

Which produces this:

'(root "I want to attend " (big "RacketCon " (strong "this") " year") ".")

Of course you can also nest X-expressions in your return value:

"article.html.pm"
#lang pollen
 
◊(define (em . parts) `(extra (big ,@parts)))
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

The second option for building X-expressions is to use the txexpr: Tagged X-expressions library that’s included with Pollen (see those docs for more information).

7.4.2.4 Interpolating variables into strings

The usual way is to use the format function:

(format "String with variable: ~a" variable-name)

See the docs for format and fprintf for your options.

Be careful if you’re working with integers and X-expressions — a raw integer is treated as a character code, not an integer string. Using format is essential:

Examples:

> (->html '(div "A raw integer indicates a character code: " 42))

"<div>A raw integer indicates a character code: &#42;</div>"

> (->html `(div "Use format to make it a string: " ,(format "~a" 42)))

"<div>Use format to make it a string: 42</div>"

7.4.2.5 Parsing attributes

Detecting attributes in an argument list can be tricky because a) the tag may or may not have attributes, b) those attributes may be in standard or abbreviated syntax. For this reason, Pollen provides a split-attributes function (in the pollen/tag librar) that you can use in custom tag functions to separate the attributes and elements:

"article.html.pm"
#lang pollen
 
◊(require pollen/tag)
 
◊(define (em . parts)
  (define-values (attributes elements) (split-attributes parts))
  `(extra ,attributes (big ,@elements)))
 
I want to attend ◊em['key: "value"]{RacketCon}.

This will move the elements inside the big tag, and attach the attributes to the extra tag:

'(root "I want to attend " (extra ((key "value")) (big "RacketCon")) ".")

7.5 Intermission

That was a lot of heavy material. But it also covered the most essential idea in Pollen: that every tag is a function. Congratulations on making it this far.

The good news is that the rest of this tutorial will feel more relaxed, as we put these new principles to work.

Sorry that this tutorial is longer than the others, but truly — this is the stuff that makes Pollen different. If you’re not feeling enthusiastic by now, you should bail out.

Otherwise, get ready to rock.

7.6 Organizing functions

In the tag-function examples so far, we’ve defined each function within the source file where we used it. This is fine for quick little functions.

But more often, you’re going to want to use functions defined elsewhere, and store your own functions available so they’re available to your source files.

For now, we’re just invoking functions within a Pollen markup file. But as you’ll see in the fourth tutorial, any function can be called from any kind of Pollen source file.

7.6.1 Using Racket’s function libraries

Any function in Racket’s extensive libraries can be called by loading the library with the require command, which will make all its functions and constants available with the usual Pollen command syntax:

"article.html.pm"
#lang pollen
 
◊(require racket/math)
 
Pi is close to ◊|pi|.
The hyperbolic sine of pi is close to ◊(sinh pi).

The result:

'(root "Pi is close to " 3.141592653589793 "." "\n" "The hyperbolic sine of pi is close to " 11.548739357257748 ".")

One caveat — you’re still in a Pollen markup file, so the return value of whatever function you call has to produce a string or an X-expression, so it can be merged into the document. This is similar to the restriction introduced in the first tutorial where functions used in preprocessor files had to produce text. -Pollen won’t stop you from calling a function that returns an incompatible value, like plot, which returns a bitmap image:

"article.html.pm"
#lang pollen
 
◊(require math plot)
 
Here's a sine wave:
◊(plot (function sin (- pi) pi #:label "y = sin(x)"))

But it won’t work when you try to run it in DrRacket or load it in the project server.

It would be fine, however, to call a different kind of plot function that returned an SVG result, because any XML-ish data structure can be converted to an X-expression.

Super web nerds also know that binary data can be converted into XML-ish form by encoding the file as a base-64 data URL — but if you know what I’m talking about, then you don’t need my help to try it.

For functions that don’t return a string or an X-expression, you can always make a conversion by hand. For instance, consider range, a Racket function that returns a list of integers:

"article.html.pm"
#lang pollen
 
◊(require racket/list)
A list of integers: ◊(range 5)

This will produce an error in DrRacket:

pollen markup error: in '(root "A list of integers: " (0 1 2 3 4)), '(0 1 2 3 4) is not a valid element (must be txexpr, string, symbol, XML char, or cdata)

In a case like this, you can explicitly convert the return value to a string (in whatever way makes sense):

"article.html.pm"
#lang pollen
 
◊(require racket/list racket/string)
A list of integers: ◊(string-join (map number->string (range 5)))

And get this output:

'(root "A list of integers: " "0 1 2 3 4")

7.6.2 The directory-require.rkt file

Don’t skip this section! It explains a concept that’s essential to understanding how Pollen works.

As you get more comfortable attaching behavior to tags using tag functions, you’ll likely want to create some functions that can be shared between multiple source files. The directory-require.rkt file is a special file that is automatically imported by Pollen source files in the same directory. So every function and value provided by directory-require.rkt can be used in these Pollen files.

First, using this file is not mandatory. You can always import functions and values from another file using require (as seen in the previous section). The directory-require.rkt is just meant to cure the tedium of importing the same file into every Pollen source file in your project. In a small project, not much tedium; in a large project, more.

Second, notice from the .rkt suffix that directory-require.rkt is a source file containing Racket code, not Pollen code. This is the default because while Pollen is better for text-driven source files, Racket is better for code-driven source files. Still, the choice is yours: the name of this file can be changed by resetting the world:directory-require value.

Third, notice from the directory- prefix that directory-require.rkt is only used by Pollen source files in the same directory. So if your project has source files nested inside a subdirectory, you’ll need to explicitly create another directory-require.rkt there and share the functions & values as needed.

“Why not make this file visible throughout a project, rather than just a directory?” Good idea, but I couldn’t figure out how to do it without creating finicky new dependencies. If you have a better idea, I’m open to it.

Let’s see how this works in practice. In the same directory as article.html.pm, create a new directory-require.rkt file as follows:

"directory-require.rkt"
#lang racket
(define author "Trevor Goodchild")
(provide author)

Here we use the define function (which we’ve seen before) to set author equal to "Trevor Goodchild". Note the final step: consistent with standard Racket rules, we have to explicitly provide the new value so that other files can see it (unlike Python, things you define in Racket are private by default, not public).

Then update good old article.html.pm:

"article.html.pm"
#lang pollen
 
The author is ◊|author|.

Run this in DrRacket and you’ll get:

'(root "The author is " "Trevor Goodchild" ".")

Now, in the same dirctory, create a second Pollen source file:

"barticle.html.pm"
#lang pollen
 
The author is really ◊|author|?

Run this, and you’ll get:

'(root "The author is really " "Trevor Goodchild" "?")

That’s all there is to it. Everything provided by directory-require.rkt is automatically available within each Pollen source file.

You can include functions, including tag functions, the same way. For instance, add a function for em:

"directory-require.rkt"
#lang racket
(define author "Trevor Goodchild")
(define (em . parts) `(extra (big ,@parts)))
(provide author em)

Then use it in a source file:

"article.html.pm"
#lang pollen
 
The ◊em{author} is ◊em{◊|author|}.

With the expected results:

'(root "The " (extra (big "author")) " is " (extra (big "Trevor Goodchild")) ".")

7.7 Putting it all together

[Coming soon]

    ← prev  up  next → 
 
\ No newline at end of file +7 Third tutorial
▼Pollen:   the book is a program
►7 Third tutorial
On this page:
7.1 Prerequisites
7.2 Pollen markup vs. XML
7.2.1 The XML problem
7.2.2 What Pollen markup does differently
7.2.3 “But I really need XML…”
7.3 Writing with Pollen markup
7.3.1 Creating a Pollen markup file
7.3.2 Tags & tag functions
7.3.3 Attributes
7.3.4 What are custom tags good for?
7.3.5 Semantic markup
7.3.6 Format independence
7.3.7 Using custom tags
7.3.8 Choosing custom tags
7.4 Tags are functions
7.4.1 Attaching behavior to tags
7.4.2 Notes for experienced programmers
7.4.2.1 Point of no return
7.4.2.2 Multiple input values & rest arguments
7.4.2.3 Returning an X-expression
7.4.2.4 Interpolating variables into strings
7.4.2.5 Parsing attributes
7.5 Intermission
7.6 Organizing functions
7.6.1 Using Racket’s function libraries
7.6.2 The directory-require.rkt file
7.7 Decoding markup via the root tag
7.8 Putting it all together
6.1.0.5
    ← prev  up  next → 

7 Third tutorial

Now you’re getting to the good stuff. In this tutorial, you’ll use Pollen to publish a multi-page article written in Pollen markup. You’ll learn about:

If you want the shortest possible introduction to Pollen, try the Quick tour.

7.1 Prerequisites

I’ll assume you’ve completed the Second tutorial and that you understand the principles of Pollen authoring mode — creating source files, converting them to X-expressions, and then combining them with templates to make output files.

Because now it’s time to pick up the pace. You’ve learned how to do some handy things with Pollen. But we haven’t yet exploited the full fusion of writing environment and programming language. I promised you that The book is a program, right? So let’s do some programming.

7.2 Pollen markup vs. XML

You can skip this section if XML holds no interest. But Pollen markup evolved out of my attempt to come up with an alternative to XML that would be more usable for writing. So if you’re familiar with XML, the contrast may be helpful.

7.2.1 The XML problem

In the Second tutorial, I made the case that Markdown is a limiting format for authors. Why? Markdown is essentially a notation system for HTML tags. As such, it has three problems: it’s not semantic, it only covers a limited subset of HTML tags, and it can’t be extended by an author.

These problems are partly limitations of HTML itself. And these limitations were meant to be cured by XML — the X stands for extensible. In principle, XML allows you to define whatever tags you like and use them in your document.

So why hasn’t XML taken over the world? In practice, XML promises more than it delivers. The reasons are apparent to any writer who’s attempted to use XML as an authoring format:

The nicest thing we could say about XML is that its intentions are good. It’s oriented toward the right goals. But its benefits are buried under atrocious ergonomics.

7.2.2 What Pollen markup does differently

Pollen markup can be seen as a way of reaping the benefits of XML without incurring the headaches. Like XML, Pollen markup allows you to freely tag your text. But unlike XML:

7.2.3 “But I really need XML…”

You can have XML. There’s nothing wrong with using Pollen markup to generate XML files that can then be fed into an existing XML processing pipeline. In other words, using Pollen markup, you can treat XML as an output format rather than an input format.

In this tutorial, I’ll be rendering Pollen markup with an HTML template. But you could easily use the same workflow with an XML template and thus end up with XML files.

7.3 Writing with Pollen markup

Pollen markup is a free-form markup system that lets you add arbitrary tags and attributes to your text. By arbitrary, I mean that they don’t need to match up with an existing schema or specification (e.g., the tags permitted by HTML). They can — but that’s an option, not a requirement.

I like to think of Pollen markup a way of capturing not just the text, but also my ideas about the text. Some of these are low-level ideas (“this text should be italicized”). Some are high-level ideas (“this text is the topic of the page”). Some are just notes to myself. In short, everything I know about the text becomes part of the text.

In so doing, Pollen markup becomes the source code of the book. Let’s try it out.

7.3.1 Creating a Pollen markup file

We’re going to use Pollen markup to make a file that will ultimately be HTML. So consistent with the authoring-mode workflow we learned in the Second tutorial, we’ll start with our desired output filename, article.html, and then append the Pollen markup suffix, .pm.

In DrRacket, start a new file called article.html.pm like so (BTW you can use any sample text you like):

"article.html.pm"
#lang pollen
 
I want to attend RacketCon this year.

Consistent with usual authoring-mode policy, when you run this file, you’ll get an X-expression that starts with root:

'(root "I want to attend RacketCon this year.")

Remember, even though the first line of the file is #lang pollen — same as the last tutorial — the new .pm suffix signals that Pollen should interpret the source as Pollen markup. Look what happens if you goof up and put Markdown source in a Pollen markup file, like so:

#lang pollen
 
I am **so** excited to attend __RacketCon__ this year.

The Markdown syntax will be ignored, and pass through to the output:

'(root "I am **so** excited to attend __RacketCon__ this year.")

Restore the non-Markdown source, and let’s continue.

7.3.2 Tags & tag functions

Pollen markup uses the same Pollen command syntax that we first saw in Adding commands. Previously, we used this command syntax to invoke functions like define and ->html. Pollen markup is used to invoke a special kind of function called a tag function, which is a function that, by default, adds a tag to the text.

To see how this works, restore your article.html.pm file to its original state:

#lang pollen
 
I want to attend RacketCon this year.

We can add any tag with Pollen markup, but for now, let’s start with an old favorite: em, which is used in HTML to add emphasis to text. We apply a tag by starting with the lozenge character (◊) followed by the tag name em, followed by the text in curly braces, like so:

"article.html.pm"
#lang pollen
 
I want to attend ◊em{RacketCon this year}.

Run this file in DrRacket and see the X-expression that results:

'(root "I want to attend " (em "RacketCon this year") ".")

You won’t be surprised to hear that you can nest tags:

"article.html.pm"
#lang pollen
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

With the expected results:

'(root "I want to attend " (em "RacketCon " (strong "this") " year") ".")

7.3.3 Attributes

Attributes are like tags for tags. Each attribute is a key–value pair where the key is any name, and the value is a string. Anyone who’s seen HTML is familiar with them:

<span class="author">Prof. Leonard</span>

Here, class is an attribute for span that has value "author". And this is what it looks like as an X-expression:

'(span ((class "author")) "Prof. Leonard")

You can add any number of attributes to a tag (first as an X-expression, then as HTML):

'(span ((class "author")(id "primary")(living "true")) "Prof. Leonard")

<span class="author" id="primary" living="true">Prof. Leonard</span>

In Pollen markup, attributes have the same logic, but a slightly different syntax. In keeping with the tag notation you just saw, the span tag is added in the usual way:

"article.html.pm"
#lang pollen
 
◊span{Prof. Leonard}

Then you have two options for adding attributes. The verbose way corresponds to how the attributes appear in the X-expression:

"article.html.pm"
#lang pollen
 
◊span['((class "author")(id "primary")(living "true"))]{Prof. Leonard}

Each key–value pair is in parentheses, and then the list of pairs is within parentheses, with a quote (') at the front that signals that the text should be used literally.

This involves some superfluous typing, however, so Pollen also supports an abbreviated syntax for attributes:

"article.html.pm"
#lang pollen
 
◊span['class:"author" 'id:"primary" 'living:"true"]{Prof. Leonard}

In this form, each attribute key starts with a quote mark ' and ends with a colon :. As before, the attribute value is in quotation marks.

Both of these forms will produce the same X-expression:

'(span ((class "author")(id "primary")(living "true")) "Prof. Leonard")

Now that you know how to make tags and attributes, you might wonder whether Pollen markup can be used as a quick & dirty HTML-notation system. Sure — for a quick & dirty project, why not. Recall that X-expressions are just alternative notation for the standard angle-bracket notation used in HTML. So if you wanted HTML like this:

<div class="red" style="font-size:150%">Important <em>News</em></div>

You could write it in Pollen markup like so:

◊div['class:"red" style:"font-size:150%"]{Important ◊em{News}}

And then just convert it (using the ->html function) into the HTML above. Thus, the tags you already know and love (?) can be used in Pollen markup, but with fewer keystrokes and cruft.

Still, if Pollen markup were just an alternative notation system for HTML tags, it would be pretty boring. As I alluded above, that’s merely a boring way to use it.

In the XML spirit, Pollen markup lets you use any tags you want. That’s considerably less boring.

7.3.4 What are custom tags good for?

XML jocks can skip this section, since you already know. But if you’ve been mired in Markdown or HTML, read on.

Tags, broadly speaking, are a means of annotating a text with extra information, which I’ll call metadata (using that term in its generic sense, not in any fiddly computery way). Metadata is the key tool that enables an author to write a book with the benefits of semantic markup and format independence.

7.3.5 Semantic markup

Semantic markup means adding metadata to text according to the meaning of the text, not merely its intended visual appearance. So rather than tagging RacketCon with an em tag, as we did above to indicate how the word should look, maybe we would tag it with an event tag, to indicate what kind of thing it is.

Semantic markup lets an author specify distinctions that would be ambiguous in pure visual terms, thereby capturing more meaning and intent. For instance, in books, italic styling is commonly applied to a number of unrelated types of information: emphasized words, movie titles, terms being used for the first time, headings, captions and labels, and so on. Under a non-semantic formatting scheme, perhaps one would tag them all em. But in semantic terms, one would tag them movie-title, first-use, heading, as appropriate.

This has two major benefits. First, by separating appearance and meaning, an author can manage the content of the book in useful ways. For instance, if every movie title were tagged as movie-title rather than italic, then it would be simple to generate a list of all movies mentioned in the book (for the author’s benefit) or a page index of movie references (for the reader’s benefit). But without that semantic tagging, a movie title couldn’t be distinguished from any other italicized text.

7.3.6 Format independence

The second benefit of custom tags is format independence, or the ability to change the rendering of the text to suit a particular device or context.

When a text is encrusted with format-specific visual tags — for instance, HTML tags — then the document markup is entangled with a single output format. If you only need one output format, fine.

But increasingly, book authors have been called upon to publish their work in multiple formats: paper and PDF, but also web, e-book, or other natively digital formats, that connect to devices with differing display capabilities.

Yes, I know that many of these formats are based on variants of HTML. But the HTML you can use in a desktop web browser is quite different from, say, the HTML you can use in a Kindle .mobi file. The .mobi file has other technical requirements too, like an .ncx and .opf file. So despite some genetic kinship, these HTML-ish formats are best understood as separate targets.

Using a display-driven model to manage this complexity is a terrible idea — as anyone who’s tried it can attest. Converting from one display-based file type to another — for instance, word processor to HTML, or HTML to PDF — is an exercise in frustration and drain-circling expectations.

This isn’t surprising. For a long time, text processing has been dominated by this display-driven model. Most word processors, like Microsoft Word and Pages, have been built around this model. It worked well enough in the era where most documents were eventually going to be printed on paper (or a paper simulator like PDF). HTML was a technical leap forward, but not a conceptual leap: it mostly represented the display options available in a web browser.

There’s a couple TeX fans at the back of the room, waving their arms. Yes, TeX got a lot of things right. In practice, however, it never became a core tool for electronic publishing (which, to be fair, didn’t exist when TeX was written). Plenty of ideas in Pollen were lifted from TeX.

For a document to be format independent, two conditions have to be satisfied.

First, the document has to be readable by other programs, so they can handle the conversion of format-independent markup into a format-specific rendering (e.g., mapping semantic tags like movie-title onto visual tags like em). Most word-processor formats, like Word’s .docx, are bad for authoring because these formats are opaque and proprietary. We needn’t get into the political objections. As a practical matter, they’re inarguably restrictive — if you can’t get your data out of your file, you’re stuck.

Second, the document itself has to be represented in a way that’s independent of the particularities of any one format. For instance, HTML is a bad authoring format because it encourages authors to litter their text with HTML-isms like h1 and span. These have no meaning outside of HTML, and thus will always cause conversion problems. The same goes for Markdown, which is simply HTML in disguise.

The solution to the first condition is to use text-based markup rather than proprietary file types. The solution to the second condition is to let authors define custom tags for the document, rather than the other way around. Pollen markup incorporates both of these ideas.

7.3.7 Using custom tags

You can insert a custom tag using the same syntax as any other tag. Suppose you want to use an event tag to mark events. You would insert it like so:

"article.html.pm"
#lang pollen
 
I want to attend ◊event{RacketCon} this year.

This markup will turn into this X-expression:

'(root "I want to attend " (event "RacketCon") " this year.")

Which is equivalent to this XML:

<root>I want to attend <event>RacketCon</event> this year.</root>

In truth, Pollen doesn’t notice any difference between a custom tag vs. a standard HTML tag vs. any other kind of tag. They’re all just markup tags. If you want to restrict yourself to a certain vocabulary of tags, you can. If you want to set up Pollen to enforce those restrictions, you can do that too. But by default, Pollen doesn’t impose restrictions like this. In general, you can pick any tag name you want, and it will work.

Don’t take my word for it. See what happens if you write this:

"article.html.pm"
#lang pollen
 
I want to attend ◊verylongandimpracticaltagname{RacketCon} this year.

One small but important exception to this rule. If you were wondering why I sometimes call them tag functions instead of just tags, it’s because under the hood, every tag is implemented as a function. The default behavior of this function is just to wrap the text in a tag with the given name.

The benefit of treating tags as functions will become evident later in this tutorial. But the cost of this approach is that tags occupy the same namespace as the other functions available in Pollen (and by extension, Racket). So if you try to use a tag name that’s already the name of an existing function, an error will occur.

For instance, let’s suppose you try to use a custom tag called length:

"article.html.pm"
#lang pollen
 
The Panama Canal is ◊length{77km} across.

When you run this file, you get an error:

length: contract violation;
expected: list?
  given: "77km"

The problem is that Racket already provides a function called length. Consistent with the usual rules of Pollen command notation, your command is interpreted as an attempt to invoke the length function, rather than apply a tag named length.

In practice, namespace clashes are rare. But if necessary, they’re easy to work around (for the simplest method, see Invoking tag functions).

7.3.8 Choosing custom tags

You just saw that using custom tags is easy. Choosing custom tags, on the other hand, is less science than art. As the author, it’s up to you. Some guidelines:

And most important:

As we’ll see in the next section, this is where your book truly becomes programmable.

7.4 Tags are functions

Don’t skip this section! It explains a concept that’s essential to understanding how Pollen works.

If you’ve used HTML or XML, tags are just tags: things you type into the document that look the same going out as they did going in. Tags can be used to select document elements or assign styling (via CSS). But they don’t have any deeper effect on the document content.

That’s not so in Pollen. Under the hood, Pollen is just an alternate way of writing code in the Racket programming language. And tags, instead of being inert markers, are actually functions.

I think most of you know what a function is, but just to be safe — in programming, a function is a chunk of code that accepts some input, processes it, and then returns a value. Asking a function to process some data is known as calling the function.

Leading us to the Three Golden Rules of Pollen Tags:

  1. Every Pollen tag calls a function with the same name.

  2. The input values for that function are the attributes and content of the tag.

  3. The whole tag — tag name, attributes, and content — is replaced with the return value of the called function.

You’ve already seen the simplest kind of function in a Pollen document: the default tag function, which emulates the behavior of standard markup tags.

Let’s revisit an earlier example, now with the help of the Golden Rules:

"article.html.pm"
#lang pollen
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

What happens when you run this source? Working from the inside out, Pollen calls the function strong with the input "this". The result is (strong "this"). Then Pollen calls the function em with the three input values "RacketCon " (strong "this") " year", which yields (em "RacketCon " (strong "this") " year"). Finally, Pollen calls the root function with everything in the document, resulting in:

'(root "I want to attend " (em "RacketCon " (strong "this") " year") ".")

7.4.1 Attaching behavior to tags

Sometimes this default behavior will suffice. But other times, you’ll want to change the behavior of a tag. Why? Here are some useful examples of what you, as an author, can do with custom tag functions:

Having invited you to gaze across these vistas, I apologize that my example here in this tutorial is necessarily tip-of-the-iceberg. I’ll be adding a more detailed guide to writing Pollen functions, both simple and crafty.

How do you change the behavior of a tag? By 1) writing a new function and 2) giving it the name of the tag. Once you do this, this new behavior will automatically be invoked when you use the tag.

For example, let’s redefine the strong tag in our example above to simply print BOOM:

"article.html.pm"
#lang pollen
 
◊define[(strong . lines)]{BOOM}
 
I want to attend ◊em{RacketCon ◊strong{this} year}

When you run this file, you indeed get:

'(root "I want to attend " (em "RacketCon " "BOOM" " year"))

How does this work? First, although you can define a function in Pollen command syntax using either of The two command modes: text mode & Racket mode, it tends to be easier to use Racket mode. I wrote the first one in text mode. But for clarity, I’m going to switch to Racket mode (run this file and convince yourself it comes out the same):

"article.html.pm"
#lang pollen
 
◊(define (strong word) "BOOM")
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

Let’s look at our new function definition. As usual, we start with the lozenge character (◊) to denote a Pollen command. Then we use define to introduce a function definition. The name of the function comes next, which needs to match our tag name, strong. The expression (strong word) means “the name of this function is strong, and it takes a single word as input, which we’ll refer to as word.” Finally we have the return value, which is "BOOM".

Let’s run this file again, but go back to the Golden Rules to understand what happens. Working from the inside out, Pollen calls the function strong with the input "this" — same as before. But this time, the result of the strong function is not (strong "this"), but simply BOOM. Then Pollen calls the function em with the three input values "RacketCon " "BOOM" " year", which yields (em "RacketCon " "BOOM" " year"). Finally, Pollen calls the root function with everything in the document, resulting in:

'(root "I want to attend " (em "RacketCon " "BOOM" " year"))

This example is contrived, of course. But the basic idea — defining a function with the name of a tag — is the foundation of programmability in Pollen. If you get this, and the Golden Rules, you get everything.

7.4.2 Notes for experienced programmers

Having said that, some of you are probably eager to hack around a bit. Let me chip off a few more cubes from the iceberg to help you on your way. (Everyone else, take five.)

7.4.2.1 Point of no return

If you’ve written functions in other programming languages, you might be accustomed to using a return statement to send a value back from the function. This doesn’t exist in Pollen or Racket — the return value of any function is just the last expression evaluated. In the example below, "BAP" becomes the return value because it’s in the last position, and "BOOM" is ignored:

"article.html.pm"
#lang pollen
 
◊(define (strong word) "BOOM" "BAP")
 
I want to attend ◊em{RacketCon ◊strong{this} year}.
7.4.2.2 Multiple input values & rest arguments

Sometimes a tag will have only one word or string that becomes its input. More likely, however, it will have multiple values (this is inevitable with nested tags, because the results aren’t concatenated). For instance, if we attach our function to em rather than strong:

"article.html.pm"
#lang pollen
 
◊(define (em word) "BOOM")
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

Look what happens:

em: arity mismatch;
the expected number of arguments does not match the given number
expected: 1
  given: 3

The error arises because the em function is getting three arguments — "RacketCon " "BOOM" " year" — but has been defined to only accept one argument, word. This is the “arity mismatch.”

To fix this, it’s better to get in the habit of writing tag functions that accept an indefinite number of input values. You do this by defining your function with a rest argument (as in, “give me the rest of the input values.”) To use a rest argument, put it last in your list of input arguments, and add a period . before:

"article.html.pm"
#lang pollen
 
◊(define (em . parts) "BOOM")
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

This time, the source file will run without an error, producing this:

'(root "I want to attend " "BOOM" ".")

A rest argument like parts is a list of individual arguments. So if you want to unpack & process these arguments separately, you can use Racket’s extensive list-processing functions (see Pairs and Lists). Also see quasiquote below.

7.4.2.3 Returning an X-expression

Often, you won’t use a tag function to replace a whole tag with a string — you’ll replace it with a different tag, described by an X-expression, like so:

"article.html.pm"
#lang pollen
 
◊(define (em . parts) '(big "BOOM"))
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

Which produces:

'(root "I want to attend " (big "BOOM") ".")

The quote mark ' before the X-expression signals to Racket that you want to use what follows as a literal value.

To build X-expressions that are more elaborate, you have two options.

First is quasiquote. Quasiquote works like quote, but starts with a backtick character `. What makes it “quasi” is that you can insert variables using the unquote operator, which is a comma , or merge a list of values with the unquote-splicing operator, which is a comma followed by an @ sign ,@.

Let’s adapt the example above to use quasiquote. Suppose we want to take the parts we get as input and put them inside a big tag. This is easy to notate with quasiquote and the unquote-splicing operator, because parts is a list:

"article.html.pm"
#lang pollen
 
◊(define (em . parts) `(big ,@parts))
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

Which produces this:

'(root "I want to attend " (big "RacketCon " (strong "this") " year") ".")

Of course you can also nest X-expressions in your return value:

"article.html.pm"
#lang pollen
 
◊(define (em . parts) `(extra (big ,@parts)))
 
I want to attend ◊em{RacketCon ◊strong{this} year}.

The second option for building X-expressions is to use the txexpr: Tagged X-expressions library that’s included with Pollen (see those docs for more information).

7.4.2.4 Interpolating variables into strings

The usual way is to use the format function:

(format "String with variable: ~a" variable-name)

See the docs for format and fprintf for your options.

Be careful if you’re working with integers and X-expressions — a raw integer is treated as a character code, not an integer string. Using format is essential:

Examples:

> (->html '(div "A raw integer indicates a character code: " 42))

"<div>A raw integer indicates a character code: &#42;</div>"

> (->html `(div "Use format to make it a string: " ,(format "~a" 42)))

"<div>Use format to make it a string: 42</div>"

7.4.2.5 Parsing attributes

Detecting attributes in an argument list can be tricky because a) the tag may or may not have attributes, b) those attributes may be in standard or abbreviated syntax. For this reason, Pollen provides a split-attributes function (in the pollen/tag librar) that you can use in custom tag functions to separate the attributes and elements:

"article.html.pm"
#lang pollen
 
◊(require pollen/tag)
 
◊(define (em . parts)
  (define-values (attributes elements) (split-attributes parts))
  `(extra ,attributes (big ,@elements)))
 
I want to attend ◊em['key: "value"]{RacketCon}.

This will move the elements inside the big tag, and attach the attributes to the extra tag:

'(root "I want to attend " (extra ((key "value")) (big "RacketCon")) ".")

7.5 Intermission

That was a lot of heavy material. But it also covered the most essential idea in Pollen: that every tag is a function. Congratulations on making it this far.

The good news is that the rest of this tutorial will feel more relaxed, as we put these new principles to work.

Sorry that this tutorial is longer than the others, but truly — this is the stuff that makes Pollen different. If you’re not feeling enthusiastic by now, you should bail out.

Otherwise, get ready to rock.

7.6 Organizing functions

In the tag-function examples so far, we’ve defined each function within the source file where we used it. This is fine for quick little functions.

But more often, you’re going to want to use functions defined elsewhere, and store your own functions available so they’re available to your source files.

For now, we’re just invoking functions within a Pollen markup file. But as you’ll see in the fourth tutorial, any function can be called from any kind of Pollen source file.

7.6.1 Using Racket’s function libraries

Any function in Racket’s extensive libraries can be called by loading the library with the require command, which will make all its functions and constants available with the usual Pollen command syntax:

"article.html.pm"
#lang pollen
 
◊(require racket/math)
 
Pi is close to ◊|pi|.
The hyperbolic sine of pi is close to ◊(sinh pi).

The result:

'(root "Pi is close to " 3.141592653589793 "." "\n" "The hyperbolic sine of pi is close to " 11.548739357257748 ".")

One caveat — you’re still in a Pollen markup file, so the return value of whatever function you call has to produce a string or an X-expression, so it can be merged into the document. This is similar to the restriction introduced in the first tutorial where functions used in preprocessor files had to produce text. +Pollen won’t stop you from calling a function that returns an incompatible value, like plot, which returns a bitmap image:

"article.html.pm"
#lang pollen
 
◊(require math plot)
 
Here's a sine wave:
◊(plot (function sin (- pi) pi #:label "y = sin(x)"))

But it won’t work when you try to run it in DrRacket or load it in the project server.

It would be fine, however, to call a different kind of plot function that returned an SVG result, because any XML-ish data structure can be converted to an X-expression.

Super web nerds also know that binary data can be converted into XML-ish form by encoding the file as a base-64 data URL — but if you know what I’m talking about, then you don’t need my help to try it.

For functions that don’t return a string or an X-expression, you can always make a conversion by hand. For instance, consider range, a Racket function that returns a list of integers:

"article.html.pm"
#lang pollen
 
◊(require racket/list)
A list of integers: ◊(range 5)

This will produce an error in DrRacket:

pollen markup error: in '(root "A list of integers: " (0 1 2 3 4)), '(0 1 2 3 4) is not a valid element (must be txexpr, string, symbol, XML char, or cdata)

In a case like this, you can explicitly convert the return value to a string (in whatever way makes sense):

"article.html.pm"
#lang pollen
 
◊(require racket/list racket/string)
A list of integers: ◊(string-join (map number->string (range 5)))

And get this output:

'(root "A list of integers: " "0 1 2 3 4")

7.6.2 The directory-require.rkt file

Don’t skip this section! It explains a concept that’s essential to understanding how Pollen works.

As you get more comfortable attaching behavior to tags using tag functions, you’ll likely want to create some functions that can be shared between multiple source files. The directory-require.rkt file is a special file that is automatically imported by Pollen source files in the same directory. So every function and value provided by directory-require.rkt can be used in these Pollen files.

First, using this file is not mandatory. You can always import functions and values from another file using require (as seen in the previous section). The directory-require.rkt is just meant to cure the tedium of importing the same file into every Pollen source file in your project. In a small project, not much tedium; in a large project, more.

Second, notice from the .rkt suffix that directory-require.rkt is a source file containing Racket code, not Pollen code. This is the default because while Pollen is better for text-driven source files, Racket is better for code-driven source files. Still, the choice is yours: the name of this file can be changed by resetting the world:directory-require value.

Third, notice from the directory- prefix that directory-require.rkt is only used by Pollen source files in the same directory. So if your project has source files nested inside a subdirectory, you’ll need to explicitly create another directory-require.rkt there and share the functions & values as needed.

“Why not make this file visible throughout a project, rather than just a directory?” Good idea, but I couldn’t figure out how to do it without creating finicky new dependencies. If you have a better idea, I’m open to it.

Let’s see how this works in practice. In the same directory as article.html.pm, create a new directory-require.rkt file as follows:

"directory-require.rkt"
#lang racket
(define author "Trevor Goodchild")
(provide author)

Here we use the define function (which we’ve seen before) to set author equal to "Trevor Goodchild". Note the final step: consistent with standard Racket rules, we have to explicitly provide the new value so that other files can see it (unlike Python, things you define in Racket are private by default, not public).

Then update good old article.html.pm:

"article.html.pm"
#lang pollen
 
The author is ◊|author|.

Run this in DrRacket and you’ll get:

'(root "The author is " "Trevor Goodchild" ".")

Now, in the same dirctory, create a second Pollen source file:

"barticle.html.pm"
#lang pollen
 
The author is really ◊|author|?

Run this, and you’ll get:

'(root "The author is really " "Trevor Goodchild" "?")

That’s all there is to it. Everything provided by directory-require.rkt is automatically available within each Pollen source file.

You can include functions, including tag functions, the same way. For instance, add a function for em:

"directory-require.rkt"
#lang racket
(define author "Trevor Goodchild")
(define (em . parts) `(extra (big ,@parts)))
(provide author em)

Then use it in a source file:

"article.html.pm"
#lang pollen
 
The ◊em{author} is ◊em{◊|author|}.

With the expected results:

'(root "The " (extra (big "author")) " is " (extra (big "Trevor Goodchild")) ".")

7.7 Decoding markup via the root tag

As you’ve seen, the X-expression you get when you run a Pollen markup file always starts with a node called root. You can attach a tag function to root the same way as any other tag. For instance, you could do something simple, like change the name of the output X-expression:

"article.html.pm"
#lang pollen
 
◊(define (root . elements) `(content ,@elements))
 
The ◊tt{root} tag is now called ◊tt{content}.

Resulting in:

'(content "The " (tt "root") " tag is now called " (tt "content") ".")

But unlike other tags in your document, root contains the entire content of the document. So the function you attach to root can operate on everything.

For that reason, one of the most useful things you can do with a tag function attached to root is decoding the content of the page. Decoding refers to any post-processing of content that happens after the tags within the page have been evaluated.

Decoding is a good way to automatically accomplish:

As an example, let’s take one of my favorites — linebreak and paragraph detection. In XML authoring, you have to insert every <br /> and <p> tag by hand. This is profoundly dull, clutters up the source file, and makes editing a chore.

Instead, let’s make a decoder that allows us to denote a linebreak with a single newline in the source, and a paragraph break with a double newline. Here’s some sample content with single and double newlines:

"article.html.pm"
#lang pollen
 
The first line of the first paragraph.
And a new line.
 
The second paragraph.

But without a decoder, the newlines just get passed through:

'(root "The first line of the first paragraph." "\n" "And a new line." "\n" "\n" "The second paragraph.")

When this X-expression is converted to HTML, the newlines persist:

<root>The first line of the first paragraph.\nAnd a new line.\n\nThe second paragraph.</root>

But in HTML, raw newlines are displayed as a single space. So if you view this file in the project server, you’ll see:

The first line of the first paragraph. And a new line. The second paragraph.

Not what we want.

So we need to make a decoder. To do this, we use the decode-elements function, which provides hooks to selectively process certain categories of content within the document.

decode-elements is a convenience variant of decode, which takes a full X-expression as input. Under the hood, they work the same way, so use whichever you prefer.

Add a basic decode-elements to the source file like so:

"article.html.pm"
#lang pollen
 
◊(require pollen/decode txexpr)
◊(define (root . elements)
   (make-txexpr 'root null (decode-elements elements)))
 
The first line of the first paragraph.
And a new line.
 
The second paragraph.

The make-txexpr function is a utility from the txexpr package, which is installed with Pollen. It builds a new X-expression from a tag, attribute list, and list of elements. Here, we’ll keep the tag name root, leave the attributes as null, and append our decoded list of elements.

Racket jocks: you could also write this using quasiquote and unquote-splicing syntax as `(root ,@(decode-elements elements)). The txexpr package is just a more explicit way of accomplishing the task.

If you run this file, what changes? Right — nothing. That’s because by default, both decode-elements (and decode) will let the content pass through unaltered.

We change this by giving decode-elements the name of a processing function and attaching it to the type of content we want to process. In this case, we’re in luck — the decode module already contains a detect-paragraphs function (that also detects linebreaks). We add this function using the keyword argument #:txexpr-elements-proc, which is short for “the function used to process the elements of a tagged X-expression”:

"article.html.pm"
#lang pollen
 
◊(require pollen/decode txexpr)
◊(define (root . elements)
   (make-txexpr 'root null (decode-elements elements
     #:txexpr-elements-proc detect-paragraphs)))
 
The first line of the first paragraph.
And a new line.
 
The second paragraph.

Now, when we run the file, the X-expression has changed to include two p tags and a br tag:

'(root (p "The first line of the first paragraph." (br) "And a new line.") (p "The second paragraph."))

That means when we convert to HTML, we get the tags we need:

<root><p>The first line of the first paragraph.<br />And a new line.</p><p>The second paragraph.</p></root>

So when we view this in the project server, the linebreaks and paragraph breaks are displayed correctly:

The first line of the first paragraph.
+And a new line. +

+The second paragraph.

Of course, in practice you wouldn’t put your decoding function in a single source file. You’d make it available to all your source files by putting it in directory-require.rkt. So let’s do that now:

"directory-require.rkt"
#lang racket
(require pollen/decode txexpr)
(define (root . elements)
   (make-txexpr 'root null (decode-elements elements
     #:txexpr-elements-proc detect-paragraphs)))
(provide (all-defined-out))

We’ll also restore the source of article.html.pm to its original, simplified state:

"article.html.pm"
#lang pollen
 
The first line of the first paragraph.
And a new line.
 
The second paragraph.

And the result in the project server will be the same:

The first line of the first paragraph.
+And a new line. +

+The second paragraph.

By the way, though decoding via the root tag is the most likely usage scenario, you don’t have to do it that way. Decoding is just a special kind of tag function. So you can make a decoder that only affects a certain tag within the page. Or you can make multiple decoders for different tags. The advantage of using a decoder with root is that it can affect all the content, and it will be the last tag function that gets called.

7.8 Putting it all together

[Coming soon]

    ← prev  up  next → 
 
\ No newline at end of file