From 6c686447f9ab0d5c0d007424b6879c085fa76461 Mon Sep 17 00:00:00 2001 From: Matthew Butterick Date: Tue, 3 Jun 2014 06:27:05 -0700 Subject: [PATCH] add example of 'decode' to docs --- doc/Decode.html | 2 +- doc/First_tutorial.html | 2 +- scribblings/decode.scrbl | 10 +++++++++- 3 files changed, 11 insertions(+), 3 deletions(-) diff --git a/doc/Decode.html b/doc/Decode.html index c710318..8a0c401 100644 --- a/doc/Decode.html +++ b/doc/Decode.html @@ -1,2 +1,2 @@ -9.2 Decode
►Pollen:   the book is a program
▼9 Module reference
9.1 Cache
9.2 Decode
9.3 File
9.4 Pagetree
9.5 Render
9.6 Template
9.7 Tag
9.8 Top
9.9 World
►9.2 Decode
On this page:
decode
9.2.1 Block
project-block-tags
register-block-tag
block-txexpr?
9.2.2 Typography
whitespace?
whitespace/  nbsp?
smart-quotes
smart-dashes
detect-linebreaks
detect-paragraphs
6.0.1.11
    ← prev  up  next → 

9.2 Decode

 (require pollen/decode) package: pollen

The doc export of a Pollen markup file is a simple X-expression. Decoding refers to any post-processing of this X-expression. The pollen/decode module provides tools for creating decoders.

The decode step can happen separately from the compilation of the file. But you can also attach a decoder to the markup file’s root node, so the decoding happens automatically when the markup is compiled, and thus automatically incorporated into doc. (Following this approach, you could also attach multiple decoders to different tags within doc.)

You can, of course, embed function calls within Pollen markup. But since markup is optimized for authors, decoding is useful for operations that can or should be moved out of the authoring layer.

One example is presentation and layout. For instance, detect-paragraphs is a decoder function that lets authors mark paragraphs in their source simply by using two carriage returns.

Another example is conversion of output into a particular data format. Most Pollen functions are optimized for HTML output, but one could write a decoder that targets another format.

procedure

(decode tagged-xexpr    
  [#:txexpr-tag-proc txexpr-tag-proc    
  #:txexpr-attrs-proc txexpr-attrs-proc    
  #:txexpr-elements-proc txexpr-elements-proc    
  #:block-txexpr-proc block-txexpr-proc    
  #:inline-txexpr-proc inline-txexpr-proc    
  #:string-proc string-proc    
  #:symbol-proc symbol-proc    
  #:valid-char-proc valid-char-proc    
  #:cdata-proc cdata-proc    
  #:exclude-tags tags-to-exclude]) → txexpr?
  tagged-xexpr : txexpr?
  txexpr-tag-proc : (txexpr-tag? . -> . txexpr-tag?)
   = (λ(tag) tag)
  txexpr-attrs-proc : (txexpr-attrs? . -> . txexpr-attrs?)
   = (λ(attrs) attrs)
  txexpr-elements-proc : (txexpr-elements? . -> . txexpr-elements?)
   = (λ(elements) elements)
  block-txexpr-proc : (block-txexpr? . -> . xexpr?) = (λ(tx) tx)
  inline-txexpr-proc : (txexpr? . -> . xexpr?) = (λ(tx) tx)
  string-proc : (string? . -> . xexpr?) = (λ(str) str)
  symbol-proc : (symbol? . -> . xexpr?) = (λ(sym) sym)
  valid-char-proc : (valid-char? . -> . xexpr?) = (λ(vc) vc)
  cdata-proc : (cdata? . -> . xexpr?) = (λ(cdata) cdata)
  tags-to-exclude : (listof symbol?) = null
Recursively process a tagged-xexpr, usually the one exported from a Pollen source file as doc.

This function doesn’t do much on its own. Rather, it provides the hooks upon which harder-working functions can be hung.

Recall from [future link: Pollen mechanics] that any tag can have a function attached to it. By default, the tagged-xexpr from a source file is tagged with root. So the typical way to use decode is to attach your decoding functions to it, and then define root to invoke your decode function. Then it will be automatically applied to every doc during compile.

For instance, here’s how decode is attached to root in Butterick’s Practical Typography. There’s not much to it —

[update with actual code]

This illustrates another important point: even though decode presents an imposing list of arguments, you’re unlikely to use all of them at once. These represent possibilities, not requirements. For instance, let’s see what happens when decode is invoked without any of its optional arguments.

Examples:

> (define tx '(root "I wonder" (em "why") "this works."))
> (decode tx)

'(root "I wonder" (em "why") "this works.")

Right — nothing. That’s because the default value for the decoding arguments is the identity function, (λ (x) x). So all the input gets passed through intact unless another action is specified.

The *-proc arguments of decode take procedures that are applied to specific categories of elements within txexpr.

The txexpr-tag-proc argument is a procedure that handles X-expression tags.

Examples:

> (define tx '(p "I'm from a strange" (strong "namespace")))
; Tags are symbols, so a tag-proc should return a symbol
> (decode tx #:txexpr-tag-proc (λ(t) (string->symbol (format "ns:~a" t))))

'(ns:p "I'm from a strange" (ns:strong "namespace"))

The txexpr-attrs-proc argument is a procedure that handles lists of X-expression attributes. (The txexpr module, included at no extra charge with Pollen, includes useful helper functions for dealing with these attribute lists.)

Examples:

> (define tx '(p [[id "first"]] "If I only had a brain."))
; Attrs is a list, so cons is OK for simple cases
> (decode tx #:txexpr-attrs-proc (λ(attrs) (cons '[class "PhD"] attrs)))

'(p ((class "PhD") (id "first")) "If I only had a brain.")

Note that txexpr-attrs-proc will change the attributes of every tagged X-expression, even those that don’t have attributes. This is useful, because sometimes you want to add attributes where none existed before. But be careful, because the behavior may make your processing function overinclusive.

Examples:

> (define tx '(div (p [[id "first"]] "If I only had a brain.")
  (p "Me too.")))
; This will insert the new attribute everywhere
> (decode tx #:txexpr-attrs-proc (λ(attrs) (cons '[class "PhD"] attrs)))

'(div

  ((class "PhD"))

  (p ((class "PhD") (id "first")) "If I only had a brain.")

  (p ((class "PhD")) "Me too."))

; This will add the new attribute only to non-null attribute lists
> (decode tx #:txexpr-attrs-proc
  (λ(attrs) (if (null? attrs) attrs (cons '[class "PhD"] attrs))))

'(div (p ((class "PhD") (id "first")) "If I only had a brain.") (p "Me too."))

The txexpr-elements-proc argument is a procedure that operates on the list of elements that represents the content of each tagged X-expression. Note that each element of an X-expression is subject to two passes through the decoder: once now, as a member of the list of elements, and also later, through its type-specific decoder (i.e., string-proc, symbol-proc, and so on).

Examples:

> (define tx '(div "Double" "\n" "toil" amp "trouble"))
; Every element gets doubled ...
> (decode tx #:txexpr-elements-proc (λ(es) (append-map (λ(e) `(,e ,e)) es)))

'(div "Double" "Double" "\n" "\n" "toil" "toil" amp amp "trouble" "trouble")

; ... but only strings get capitalized
> (decode tx #:txexpr-elements-proc (λ(es) (append-map (λ(e) `(,e ,e)) es))
  #:string-proc (λ(s) (string-upcase s)))

'(div "DOUBLE" "DOUBLE" "\n" "\n" "TOIL" "TOIL" amp amp "TROUBLE" "TROUBLE")

So why do you need txexpr-elements-proc? Because some types of element decoding depend on context, thus it’s necessary to handle the elements as a group. For instance, the doubling function above, though useless, requires handling the element list as a whole, because elements are being added.

A more useful example: paragraph detection. The behavior is not merely a map across each element:

Examples:

> (define (paras tx) (decode tx #:txexpr-elements-proc detect-paragraphs))
; Context matters. Trailing whitespace is ignored ...
> (paras '(body "The first paragraph." "\n\n"))

'(body "The first paragraph.")

; ... but whitespace between strings is converted to a break.
> (paras '(body "The first paragraph." "\n\n" "And another."))

'(body (p "The first paragraph.") (p "And another."))

; A combination of both types
> (paras '(body "The first paragraph." "\n\n" "And another." "\n\n"))

'(body (p "The first paragraph.") (p "And another."))

The block-txexpr-proc argument and the inline-txexpr-proc arguments are procedures that operate on tagged X-expressions. If the X-expression meets the block-txexpr? test, it is processed by block-txexpr-proc. Otherwise, it is processed by inline-txexpr-proc. Thus every tagged X-expression will be handled by one or the other. Of course, if you want block and inline elements to be handled the same way, you can set block-txexpr-proc and inline-txexpr-proc to be the same procedure.

Examples:

> (define tx '(div "Please" (em "mind the gap") (h1 "Tuesdays only")))
> (define add-ns (λ(tx) (make-txexpr
      (string->symbol (format "ns:~a" (get-tag tx)))
      (get-attrs tx)
      (get-elements tx))))
; div and h1 are block elements, so this will only affect them
> (decode tx #:block-txexpr-proc add-ns)

'(ns:div "Please" (em "mind the gap") (ns:h1 "Tuesdays only"))

; em is an inline element, so this will only affect it
> (decode tx #:inline-txexpr-proc add-ns)

'(div "Please" (ns:em "mind the gap") (h1 "Tuesdays only"))

; this will affect all elements
> (decode tx #:block-txexpr-proc add-ns #:inline-txexpr-proc add-ns)

'(ns:div "Please" (ns:em "mind the gap") (ns:h1 "Tuesdays only"))

The string-proc, symbol-proc, valid-char-proc, and cdata-proc arguments are procedures that operate on X-expressions that are strings, symbols, valid-chars, and CDATA, respectively. Deliberately, the output contracts for these procedures accept any kind of X-expression (meaning, the procedure can change the X-expression type).

Examples:

; A div with string, entity, character, and cdata elements
> (define tx `(div "Moe" amp 62 ,(cdata #f #f "3 > 2;")))
> (define rulify (λ(x) '(hr)))
; The rulify function is selectively applied to each
> (print (decode tx #:string-proc rulify))

'(div (hr) amp 62 #(struct:cdata #f #f "3 > 2;"))

> (print (decode tx #:symbol-proc rulify))

'(div "Moe" (hr) 62 #(struct:cdata #f #f "3 > 2;"))

> (print (decode tx #:valid-char-proc rulify))

'(div "Moe" amp (hr) #(struct:cdata #f #f "3 > 2;"))

> (print (decode tx #:cdata-proc rulify))

'(div "Moe" amp 62 (hr))

Finally, the tags-to-exclude argument is a list of tags that will be exempted from decoding. Though you could get the same result by testing the input within the individual decoding functions, that’s tedious and potentially slower.

Examples:

> (define tx '(p "I really think" (em "italics") "should be lowercase."))
> (decode tx #:string-proc (λ(s) (string-upcase s)))

'(p "I REALLY THINK" (em "ITALICS") "SHOULD BE LOWERCASE.")

> (decode tx #:string-proc (λ(s) (string-upcase s)) #:exclude-tags '(em))

'(p "I REALLY THINK" (em "italics") "SHOULD BE LOWERCASE.")

The tags-to-exclude argument is useful if you’re decoding source that’s destined to become HTML. According to the HTML spec, material within a <style> or <script> block needs to be preserved literally. In this example, if the CSS and JavaScript blocks are capitalized, they won’t work. So exclude '(style script), and problem solved.

Examples:

> (define tx '(body (h1 [[class "Red"]] "Let's visit Planet Telex.")
  (style [[type "text/css"]] ".Red {color: green;}")
  (script [[type "text/javascript"]] "var area = h * w;")))
> (decode tx #:string-proc (λ(s) (string-upcase s)))

'(body

  (h1 ((class "Red")) "LET'S VISIT PLANET TELEX.")

  (style ((type "text/css")) ".RED {COLOR: GREEN;}")

  (script ((type "text/javascript")) "VAR AREA = H * W;"))

> (decode tx #:string-proc (λ(s) (string-upcase s))
  #:exclude-tags '(style script))

'(body

  (h1 ((class "Red")) "LET'S VISIT PLANET TELEX.")

  (style ((type "text/css")) ".Red {color: green;}")

  (script ((type "text/javascript")) "var area = h * w;"))

9.2.1 Block

Because it’s convenient, Pollen categorizes tagged X-expressions into two categories: block and inline. Why is it convenient? When using decode, you often want to treat the two categories differently. Not that you have to. But this is how you can.

parameter

(project-block-tags) → (listof txexpr-tag?)

(project-block-tags block-tags) → void?
  block-tags : (listof txexpr-tag?)
A parameter that defines the set of tags that decode will treat as blocks. This parameter is initialized with the HTML block tags, namely:

(address article aside audio blockquote body canvas dd div dl fieldset figcaption figure footer form h1 h2 h3 h4 h5 h6 header hgroup noscript ol output p pre section table tfoot ul video)

procedure

(register-block-tag tag) → void?

  tag : txexpr-tag?
Adds a tag to project-block-tags so that block-txexpr? will report it as a block, and decode will process it with block-txexpr-proc rather than inline-txexpr-proc.

Pollen tries to do the right thing without being told. But this is the rare case where you have to be explicit. If you introduce a tag into your markup that you want treated as a block, you must use this function to identify it, or you will get spooky behavior later on.

For instance, detect-paragraphs knows that block elements in the markup shouldn’t be wrapped in a p tag. So if you introduce a new block element called bloq without registering it as a block, misbehavior will follow:

Examples:

> (define (paras tx) (decode tx #:txexpr-elements-proc detect-paragraphs))
> (paras '(body "I want to be a paragraph." "\n\n" (bloq "But not me.")))

'(body (p "I want to be a paragraph.") (p (bloq "But not me.")))

; Wrong: bloq should not be wrapped

But once you register bloq as a block, order is restored:

Examples:

> (define (paras tx) (decode tx #:txexpr-elements-proc detect-paragraphs))
> (register-block-tag 'bloq)
> (paras '(body "I want to be a paragraph." "\n\n" (bloq "But not me.")))

'(body (p "I want to be a paragraph.") (bloq "But not me."))

; Right: bloq is treated as a block

If you find the idea of registering block tags unbearable, good news. The project-block-tags include the standard HTML block tags by default. So if you just want to use things like div and p and h1–h6, you’ll get the right behavior for free.

Examples:

> (define (paras tx) (decode tx #:txexpr-elements-proc detect-paragraphs))
> (paras '(body "I want to be a paragraph." "\n\n" (div "But not me.")))

'(body (p "I want to be a paragraph.") (div "But not me."))

procedure

(block-txexpr? v) → boolean?

  v : any/c
Predicate that tests whether v is a tagged X-expression, and if so, whether the tag is among the project-block-tags. If not, it is treated as inline. To adjust how this test works, use register-block-tag.

9.2.2 Typography

An assortment of typography & layout functions, designed to be used with decode. These aren’t hard to write. So if you like these, use them. If not, make your own.

procedure

(whitespace? v) → boolean?

  v : any/c
A predicate that returns #t for any stringlike v that’s entirely whitespace, but also the empty string, as well as lists and vectors that are made only of whitespace? members. Following the regexp-match convention, whitespace? does not return #t for a nonbreaking space. If you prefer that behavior, use whitespace/nbsp?.

Examples:

> (whitespace? "\n\n   ")

#t

> (whitespace? (string->symbol "\n\n   "))

#t

> (whitespace? "")

#t

> (whitespace? '("" "  " "\n\n\n" " \n"))

#t

> (define nonbreaking-space (format "~a" #\u00A0))
> (whitespace? nonbreaking-space)

#f

procedure

(whitespace/nbsp? v) → boolean?

  v : any/c
Like whitespace?, but also returns #t for nonbreaking spaces.

Examples:

> (whitespace/nbsp? "\n\n   ")

#t

> (whitespace/nbsp? (string->symbol "\n\n   "))

#t

> (whitespace/nbsp? "")

#t

> (whitespace/nbsp? '("" "  " "\n\n\n" " \n"))

#t

> (define nonbreaking-space (format "~a" #\u00A0))
> (whitespace/nbsp? nonbreaking-space)

#t

procedure

(smart-quotes str) → string?

  str : string?
Convert straight quotes in str to curly according to American English conventions.

Examples:

> (define tricky-string
  "\"Why,\" she could've asked, \"are we in O‘ahu watching 'Mame'?\"")
> (display tricky-string)

"Why," she could've asked, "are we in O‘ahu watching 'Mame'?"

> (display (smart-quotes tricky-string))

“Why,” she could’ve asked, “are we in O‘ahu watching ‘Mame’?”

procedure

(smart-dashes str) → string?

  str : string?
In str, convert three hyphens to an em dash, and two hyphens to an en dash, and remove surrounding spaces.

Examples:

> (define tricky-string "I had a few --- OK, like 6--8 --- thin mints.")
> (display tricky-string)

I had a few --- OK, like 6--8 --- thin mints.

> (display (smart-dashes tricky-string))

I had a few—OK, like 6–8—thin mints.

; Monospaced font not great for showing dashes, but you get the idea

procedure

(detect-linebreaks tagged-xexpr-elements 
  [#:separator linebreak-sep 
  #:insert linebreak]) 
 → txexpr-elements?
  tagged-xexpr-elements : txexpr-elements?
  linebreak-sep : string? = world:linebreak-separator
  linebreak : xexpr? = '(br)
Within tagged-xexpr-elements, convert occurrences of linebreak-sep ("\n" by default) to linebreak, but only if linebreak-sep does not occur between blocks (see block-txexpr?). Why? Because block-level elements automatically display on a new line, so adding linebreak would be superfluous. In that case, linebreak-sep just disappears.

Examples:

> (detect-linebreaks '(div "Two items:" "\n" (em "Eggs") "\n" (em "Bacon")))

'(div "Two items:" (br) (em "Eggs") (br) (em "Bacon"))

> (detect-linebreaks '(div "Two items:" "\n" (div "Eggs") "\n" (div "Bacon")))

'(div "Two items:" (div "Eggs") (div "Bacon"))

procedure

(detect-paragraphs elements 
  [#:separator paragraph-sep 
  #:tag paragraph-tag 
  #:linebreak-proc linebreak-proc]) 
 → txexpr-elements?
  elements : txexpr-elements?
  paragraph-sep : string? = world:paragraph-separator
  paragraph-tag : symbol? = 'p
  linebreak-proc : (txexpr-elements? . -> . txexpr-elements?)
   = detect-linebreaks
Find paragraphs within elements, as denoted by paragraph-sep, and wrap them with paragraph-tag, unless the element is already a block-txexpr? (because in that case, the wrapping is superfluous). Thus, as a consequence, if paragraph-sep occurs between two blocks, it’s ignored.

The paragraph-tag argument sets the tag used to wrap paragraphs.

The linebreak-proc argument allows you to use a different linebreaking procedure other than the usual detect-linebreaks.

Examples:

> (detect-paragraphs '("First para" "\n\n" "Second para"))

'((p "First para") (p "Second para"))

> (detect-paragraphs '("First para" "\n\n" "Second para" "\n" "Second line"))

'((p "First para") (p "Second para" (br) "Second line"))

> (detect-paragraphs '("First para" "\n\n" (div "Second block")))

'((p "First para") (div "Second block"))

> (detect-paragraphs '((div "First block") "\n\n" (div "Second block")))

'((div "First block") (div "Second block"))

> (detect-paragraphs '("First para" "\n\n" "Second para") #:tag 'ns:p)

'((ns:p "First para") (ns:p "Second para"))

> (detect-paragraphs '("First para" "\n\n" "Second para" "\n" "Second line")
  #:linebreak-proc (λ(x) (detect-linebreaks x #:insert '(newline))))

'((p "First para") (p "Second para" (newline) "Second line"))

    ← prev  up  next → 
 
\ No newline at end of file +9.2 Decode
►Pollen:   the book is a program
▼9 Module reference
9.1 Cache
9.2 Decode
9.3 File
9.4 Pagetree
9.5 Render
9.6 Template
9.7 Tag
9.8 Top
9.9 World
►9.2 Decode
On this page:
decode
9.2.1 Block
project-block-tags
register-block-tag
block-txexpr?
9.2.2 Typography
whitespace?
whitespace/  nbsp?
smart-quotes
smart-dashes
detect-linebreaks
detect-paragraphs
6.0.1.11
    ← prev  up  next → 

9.2 Decode

 (require pollen/decode) package: pollen

The doc export of a Pollen markup file is a simple X-expression. Decoding refers to any post-processing of this X-expression. The pollen/decode module provides tools for creating decoders.

The decode step can happen separately from the compilation of the file. But you can also attach a decoder to the markup file’s root node, so the decoding happens automatically when the markup is compiled, and thus automatically incorporated into doc. (Following this approach, you could also attach multiple decoders to different tags within doc.)

You can, of course, embed function calls within Pollen markup. But since markup is optimized for authors, decoding is useful for operations that can or should be moved out of the authoring layer.

One example is presentation and layout. For instance, detect-paragraphs is a decoder function that lets authors mark paragraphs in their source simply by using two carriage returns.

Another example is conversion of output into a particular data format. Most Pollen functions are optimized for HTML output, but one could write a decoder that targets another format.

procedure

(decode tagged-xexpr    
  [#:txexpr-tag-proc txexpr-tag-proc    
  #:txexpr-attrs-proc txexpr-attrs-proc    
  #:txexpr-elements-proc txexpr-elements-proc    
  #:block-txexpr-proc block-txexpr-proc    
  #:inline-txexpr-proc inline-txexpr-proc    
  #:string-proc string-proc    
  #:symbol-proc symbol-proc    
  #:valid-char-proc valid-char-proc    
  #:cdata-proc cdata-proc    
  #:exclude-tags tags-to-exclude]) → txexpr?
  tagged-xexpr : txexpr?
  txexpr-tag-proc : (txexpr-tag? . -> . txexpr-tag?)
   = (λ(tag) tag)
  txexpr-attrs-proc : (txexpr-attrs? . -> . txexpr-attrs?)
   = (λ(attrs) attrs)
  txexpr-elements-proc : (txexpr-elements? . -> . txexpr-elements?)
   = (λ(elements) elements)
  block-txexpr-proc : (block-txexpr? . -> . xexpr?) = (λ(tx) tx)
  inline-txexpr-proc : (txexpr? . -> . xexpr?) = (λ(tx) tx)
  string-proc : (string? . -> . xexpr?) = (λ(str) str)
  symbol-proc : (symbol? . -> . xexpr?) = (λ(sym) sym)
  valid-char-proc : (valid-char? . -> . xexpr?) = (λ(vc) vc)
  cdata-proc : (cdata? . -> . xexpr?) = (λ(cdata) cdata)
  tags-to-exclude : (listof symbol?) = null
Recursively process a tagged-xexpr, usually the one exported from a Pollen source file as doc.

This function doesn’t do much on its own. Rather, it provides the hooks upon which harder-working functions can be hung.

Recall from [future link: Pollen mechanics] that any tag can have a function attached to it. By default, the tagged-xexpr from a source file is tagged with root. So the typical way to use decode is to attach your decoding functions to it, and then define root to invoke your decode function. Then it will be automatically applied to every doc during compile.

For instance, here’s how decode is attached to root in Butterick’s Practical Typography. There’s not much to it —

(define (root . items)
  (decode (make-txexpr 'root '() items)
          #:txexpr-elements-proc detect-paragraphs
          #:block-txexpr-proc (compose1 hyphenate wrap-hanging-quotes
                insert-nonbreaking-last-space)
          #:string-proc (compose1 smart-quotes smart-dashes)
          #:exclude-tags '(style script)))

This illustrates another important point: even though decode presents an imposing list of arguments, you’re unlikely to use all of them at once. These represent possibilities, not requirements. For instance, let’s see what happens when decode is invoked without any of its optional arguments.

Examples:

> (define tx '(root "I wonder" (em "why") "this works."))
> (decode tx)

'(root "I wonder" (em "why") "this works.")

Right — nothing. That’s because the default value for the decoding arguments is the identity function, (λ (x) x). So all the input gets passed through intact unless another action is specified.

The *-proc arguments of decode take procedures that are applied to specific categories of elements within txexpr.

The txexpr-tag-proc argument is a procedure that handles X-expression tags.

Examples:

> (define tx '(p "I'm from a strange" (strong "namespace")))
; Tags are symbols, so a tag-proc should return a symbol
> (decode tx #:txexpr-tag-proc (λ(t) (string->symbol (format "ns:~a" t))))

'(ns:p "I'm from a strange" (ns:strong "namespace"))

The txexpr-attrs-proc argument is a procedure that handles lists of X-expression attributes. (The txexpr module, included at no extra charge with Pollen, includes useful helper functions for dealing with these attribute lists.)

Examples:

> (define tx '(p [[id "first"]] "If I only had a brain."))
; Attrs is a list, so cons is OK for simple cases
> (decode tx #:txexpr-attrs-proc (λ(attrs) (cons '[class "PhD"] attrs)))

'(p ((class "PhD") (id "first")) "If I only had a brain.")

Note that txexpr-attrs-proc will change the attributes of every tagged X-expression, even those that don’t have attributes. This is useful, because sometimes you want to add attributes where none existed before. But be careful, because the behavior may make your processing function overinclusive.

Examples:

> (define tx '(div (p [[id "first"]] "If I only had a brain.")
  (p "Me too.")))
; This will insert the new attribute everywhere
> (decode tx #:txexpr-attrs-proc (λ(attrs) (cons '[class "PhD"] attrs)))

'(div

  ((class "PhD"))

  (p ((class "PhD") (id "first")) "If I only had a brain.")

  (p ((class "PhD")) "Me too."))

; This will add the new attribute only to non-null attribute lists
> (decode tx #:txexpr-attrs-proc
  (λ(attrs) (if (null? attrs) attrs (cons '[class "PhD"] attrs))))

'(div (p ((class "PhD") (id "first")) "If I only had a brain.") (p "Me too."))

The txexpr-elements-proc argument is a procedure that operates on the list of elements that represents the content of each tagged X-expression. Note that each element of an X-expression is subject to two passes through the decoder: once now, as a member of the list of elements, and also later, through its type-specific decoder (i.e., string-proc, symbol-proc, and so on).

Examples:

> (define tx '(div "Double" "\n" "toil" amp "trouble"))
; Every element gets doubled ...
> (decode tx #:txexpr-elements-proc (λ(es) (append-map (λ(e) `(,e ,e)) es)))

'(div "Double" "Double" "\n" "\n" "toil" "toil" amp amp "trouble" "trouble")

; ... but only strings get capitalized
> (decode tx #:txexpr-elements-proc (λ(es) (append-map (λ(e) `(,e ,e)) es))
  #:string-proc (λ(s) (string-upcase s)))

'(div "DOUBLE" "DOUBLE" "\n" "\n" "TOIL" "TOIL" amp amp "TROUBLE" "TROUBLE")

So why do you need txexpr-elements-proc? Because some types of element decoding depend on context, thus it’s necessary to handle the elements as a group. For instance, the doubling function above, though useless, requires handling the element list as a whole, because elements are being added.

A more useful example: paragraph detection. The behavior is not merely a map across each element:

Examples:

> (define (paras tx) (decode tx #:txexpr-elements-proc detect-paragraphs))
; Context matters. Trailing whitespace is ignored ...
> (paras '(body "The first paragraph." "\n\n"))

'(body "The first paragraph.")

; ... but whitespace between strings is converted to a break.
> (paras '(body "The first paragraph." "\n\n" "And another."))

'(body (p "The first paragraph.") (p "And another."))

; A combination of both types
> (paras '(body "The first paragraph." "\n\n" "And another." "\n\n"))

'(body (p "The first paragraph.") (p "And another."))

The block-txexpr-proc argument and the inline-txexpr-proc arguments are procedures that operate on tagged X-expressions. If the X-expression meets the block-txexpr? test, it is processed by block-txexpr-proc. Otherwise, it is processed by inline-txexpr-proc. Thus every tagged X-expression will be handled by one or the other. Of course, if you want block and inline elements to be handled the same way, you can set block-txexpr-proc and inline-txexpr-proc to be the same procedure.

Examples:

> (define tx '(div "Please" (em "mind the gap") (h1 "Tuesdays only")))
> (define add-ns (λ(tx) (make-txexpr
      (string->symbol (format "ns:~a" (get-tag tx)))
      (get-attrs tx)
      (get-elements tx))))
; div and h1 are block elements, so this will only affect them
> (decode tx #:block-txexpr-proc add-ns)

'(ns:div "Please" (em "mind the gap") (ns:h1 "Tuesdays only"))

; em is an inline element, so this will only affect it
> (decode tx #:inline-txexpr-proc add-ns)

'(div "Please" (ns:em "mind the gap") (h1 "Tuesdays only"))

; this will affect all elements
> (decode tx #:block-txexpr-proc add-ns #:inline-txexpr-proc add-ns)

'(ns:div "Please" (ns:em "mind the gap") (ns:h1 "Tuesdays only"))

The string-proc, symbol-proc, valid-char-proc, and cdata-proc arguments are procedures that operate on X-expressions that are strings, symbols, valid-chars, and CDATA, respectively. Deliberately, the output contracts for these procedures accept any kind of X-expression (meaning, the procedure can change the X-expression type).

Examples:

; A div with string, entity, character, and cdata elements
> (define tx `(div "Moe" amp 62 ,(cdata #f #f "3 > 2;")))
> (define rulify (λ(x) '(hr)))
; The rulify function is selectively applied to each
> (print (decode tx #:string-proc rulify))

'(div (hr) amp 62 #(struct:cdata #f #f "3 > 2;"))

> (print (decode tx #:symbol-proc rulify))

'(div "Moe" (hr) 62 #(struct:cdata #f #f "3 > 2;"))

> (print (decode tx #:valid-char-proc rulify))

'(div "Moe" amp (hr) #(struct:cdata #f #f "3 > 2;"))

> (print (decode tx #:cdata-proc rulify))

'(div "Moe" amp 62 (hr))

Finally, the tags-to-exclude argument is a list of tags that will be exempted from decoding. Though you could get the same result by testing the input within the individual decoding functions, that’s tedious and potentially slower.

Examples:

> (define tx '(p "I really think" (em "italics") "should be lowercase."))
> (decode tx #:string-proc (λ(s) (string-upcase s)))

'(p "I REALLY THINK" (em "ITALICS") "SHOULD BE LOWERCASE.")

> (decode tx #:string-proc (λ(s) (string-upcase s)) #:exclude-tags '(em))

'(p "I REALLY THINK" (em "italics") "SHOULD BE LOWERCASE.")

The tags-to-exclude argument is useful if you’re decoding source that’s destined to become HTML. According to the HTML spec, material within a <style> or <script> block needs to be preserved literally. In this example, if the CSS and JavaScript blocks are capitalized, they won’t work. So exclude '(style script), and problem solved.

Examples:

> (define tx '(body (h1 [[class "Red"]] "Let's visit Planet Telex.")
  (style [[type "text/css"]] ".Red {color: green;}")
  (script [[type "text/javascript"]] "var area = h * w;")))
> (decode tx #:string-proc (λ(s) (string-upcase s)))

'(body

  (h1 ((class "Red")) "LET'S VISIT PLANET TELEX.")

  (style ((type "text/css")) ".RED {COLOR: GREEN;}")

  (script ((type "text/javascript")) "VAR AREA = H * W;"))

> (decode tx #:string-proc (λ(s) (string-upcase s))
  #:exclude-tags '(style script))

'(body

  (h1 ((class "Red")) "LET'S VISIT PLANET TELEX.")

  (style ((type "text/css")) ".Red {color: green;}")

  (script ((type "text/javascript")) "var area = h * w;"))

9.2.1 Block

Because it’s convenient, Pollen categorizes tagged X-expressions into two categories: block and inline. Why is it convenient? When using decode, you often want to treat the two categories differently. Not that you have to. But this is how you can.

parameter

(project-block-tags) → (listof txexpr-tag?)

(project-block-tags block-tags) → void?
  block-tags : (listof txexpr-tag?)
A parameter that defines the set of tags that decode will treat as blocks. This parameter is initialized with the HTML block tags, namely:

(address article aside audio blockquote body canvas dd div dl fieldset figcaption figure footer form h1 h2 h3 h4 h5 h6 header hgroup noscript ol output p pre section table tfoot ul video)

procedure

(register-block-tag tag) → void?

  tag : txexpr-tag?
Adds a tag to project-block-tags so that block-txexpr? will report it as a block, and decode will process it with block-txexpr-proc rather than inline-txexpr-proc.

Pollen tries to do the right thing without being told. But this is the rare case where you have to be explicit. If you introduce a tag into your markup that you want treated as a block, you must use this function to identify it, or you will get spooky behavior later on.

For instance, detect-paragraphs knows that block elements in the markup shouldn’t be wrapped in a p tag. So if you introduce a new block element called bloq without registering it as a block, misbehavior will follow:

Examples:

> (define (paras tx) (decode tx #:txexpr-elements-proc detect-paragraphs))
> (paras '(body "I want to be a paragraph." "\n\n" (bloq "But not me.")))

'(body (p "I want to be a paragraph.") (p (bloq "But not me.")))

; Wrong: bloq should not be wrapped

But once you register bloq as a block, order is restored:

Examples:

> (define (paras tx) (decode tx #:txexpr-elements-proc detect-paragraphs))
> (register-block-tag 'bloq)
> (paras '(body "I want to be a paragraph." "\n\n" (bloq "But not me.")))

'(body (p "I want to be a paragraph.") (bloq "But not me."))

; Right: bloq is treated as a block

If you find the idea of registering block tags unbearable, good news. The project-block-tags include the standard HTML block tags by default. So if you just want to use things like div and p and h1–h6, you’ll get the right behavior for free.

Examples:

> (define (paras tx) (decode tx #:txexpr-elements-proc detect-paragraphs))
> (paras '(body "I want to be a paragraph." "\n\n" (div "But not me.")))

'(body (p "I want to be a paragraph.") (div "But not me."))

procedure

(block-txexpr? v) → boolean?

  v : any/c
Predicate that tests whether v is a tagged X-expression, and if so, whether the tag is among the project-block-tags. If not, it is treated as inline. To adjust how this test works, use register-block-tag.

9.2.2 Typography

An assortment of typography & layout functions, designed to be used with decode. These aren’t hard to write. So if you like these, use them. If not, make your own.

procedure

(whitespace? v) → boolean?

  v : any/c
A predicate that returns #t for any stringlike v that’s entirely whitespace, but also the empty string, as well as lists and vectors that are made only of whitespace? members. Following the regexp-match convention, whitespace? does not return #t for a nonbreaking space. If you prefer that behavior, use whitespace/nbsp?.

Examples:

> (whitespace? "\n\n   ")

#t

> (whitespace? (string->symbol "\n\n   "))

#t

> (whitespace? "")

#t

> (whitespace? '("" "  " "\n\n\n" " \n"))

#t

> (define nonbreaking-space (format "~a" #\u00A0))
> (whitespace? nonbreaking-space)

#f

procedure

(whitespace/nbsp? v) → boolean?

  v : any/c
Like whitespace?, but also returns #t for nonbreaking spaces.

Examples:

> (whitespace/nbsp? "\n\n   ")

#t

> (whitespace/nbsp? (string->symbol "\n\n   "))

#t

> (whitespace/nbsp? "")

#t

> (whitespace/nbsp? '("" "  " "\n\n\n" " \n"))

#t

> (define nonbreaking-space (format "~a" #\u00A0))
> (whitespace/nbsp? nonbreaking-space)

#t

procedure

(smart-quotes str) → string?

  str : string?
Convert straight quotes in str to curly according to American English conventions.

Examples:

> (define tricky-string
  "\"Why,\" she could've asked, \"are we in O‘ahu watching 'Mame'?\"")
> (display tricky-string)

"Why," she could've asked, "are we in O‘ahu watching 'Mame'?"

> (display (smart-quotes tricky-string))

“Why,” she could’ve asked, “are we in O‘ahu watching ‘Mame’?”

procedure

(smart-dashes str) → string?

  str : string?
In str, convert three hyphens to an em dash, and two hyphens to an en dash, and remove surrounding spaces.

Examples:

> (define tricky-string "I had a few --- OK, like 6--8 --- thin mints.")
> (display tricky-string)

I had a few --- OK, like 6--8 --- thin mints.

> (display (smart-dashes tricky-string))

I had a few—OK, like 6–8—thin mints.

; Monospaced font not great for showing dashes, but you get the idea

procedure

(detect-linebreaks tagged-xexpr-elements 
  [#:separator linebreak-sep 
  #:insert linebreak]) 
 → txexpr-elements?
  tagged-xexpr-elements : txexpr-elements?
  linebreak-sep : string? = world:linebreak-separator
  linebreak : xexpr? = '(br)
Within tagged-xexpr-elements, convert occurrences of linebreak-sep ("\n" by default) to linebreak, but only if linebreak-sep does not occur between blocks (see block-txexpr?). Why? Because block-level elements automatically display on a new line, so adding linebreak would be superfluous. In that case, linebreak-sep just disappears.

Examples:

> (detect-linebreaks '(div "Two items:" "\n" (em "Eggs") "\n" (em "Bacon")))

'(div "Two items:" (br) (em "Eggs") (br) (em "Bacon"))

> (detect-linebreaks '(div "Two items:" "\n" (div "Eggs") "\n" (div "Bacon")))

'(div "Two items:" (div "Eggs") (div "Bacon"))

procedure

(detect-paragraphs elements 
  [#:separator paragraph-sep 
  #:tag paragraph-tag 
  #:linebreak-proc linebreak-proc]) 
 → txexpr-elements?
  elements : txexpr-elements?
  paragraph-sep : string? = world:paragraph-separator
  paragraph-tag : symbol? = 'p
  linebreak-proc : (txexpr-elements? . -> . txexpr-elements?)
   = detect-linebreaks
Find paragraphs within elements, as denoted by paragraph-sep, and wrap them with paragraph-tag, unless the element is already a block-txexpr? (because in that case, the wrapping is superfluous). Thus, as a consequence, if paragraph-sep occurs between two blocks, it’s ignored.

The paragraph-tag argument sets the tag used to wrap paragraphs.

The linebreak-proc argument allows you to use a different linebreaking procedure other than the usual detect-linebreaks.

Examples:

> (detect-paragraphs '("First para" "\n\n" "Second para"))

'((p "First para") (p "Second para"))

> (detect-paragraphs '("First para" "\n\n" "Second para" "\n" "Second line"))

'((p "First para") (p "Second para" (br) "Second line"))

> (detect-paragraphs '("First para" "\n\n" (div "Second block")))

'((p "First para") (div "Second block"))

> (detect-paragraphs '((div "First block") "\n\n" (div "Second block")))

'((div "First block") (div "Second block"))

> (detect-paragraphs '("First para" "\n\n" "Second para") #:tag 'ns:p)

'((ns:p "First para") (ns:p "Second para"))

> (detect-paragraphs '("First para" "\n\n" "Second para" "\n" "Second line")
  #:linebreak-proc (λ(x) (detect-linebreaks x #:insert '(newline))))

'((p "First para") (p "Second para" (newline) "Second line"))

    ← prev  up  next → 
 
\ No newline at end of file diff --git a/doc/First_tutorial.html b/doc/First_tutorial.html index 440bddf..d35fc42 100644 --- a/doc/First_tutorial.html +++ b/doc/First_tutorial.html @@ -3,7 +3,7 @@
standard-module-name-resolver: collection not found ...

Why? Because there’s no language called pollenxyz. Switch it back to pollen and let’s move on.

5.3.2 Putting in the text of the poem

Here’s a short, bad poem I wrote about CSS.

The margin is 42em.

The border is red.

The padding is 15em.

The border is too.

Paste the text of this poem into your DrRacket editing window, below the #lang line, so it looks like this:

#lang pollen

 

The margin is 42em.

The border is red.

The padding is 15em.

The border is too.

Run the file again. In the interactions window, you’ll see:

The margin is 8em.
The border is blue.
The padding is 2em. -
The border is too.

This shows you something important: by default, any plain text in a Pollen source file is simply printed as written when you Run the file (minus the #lang line, which is just for Racket’s benefit). If you like, edit the text of the poem and click Run again. You’ll see the updated text printed in the interactions window.

5.3.3 Saving & naming your source file

File naming in Pollen is consequential.

Ultimately, every Pollen source file in your project will be rendered into an output file. Each Pollen source file corresponds to one output file. The name of this output file will be the name of the source file minus the Pollen source extension. So a source file called file.txt.pp will become file.txt.

Thus, to build the name of a source file, we take the name we want for the output file and add the appropriate Pollen file extension. Different Pollen source files use different extensions — but more about that later. For now, the extension you’ll use for your source is .pp.

In this case, let’s say we want to end up with a file called poem.html. Therefore, the name of our source file needs to be:

the output name poem.html + the source extension .pp = poem.html.pp

(If you want to name the file something-else.html.pp, be my guest. There’s no magic associated with the prefix.)

You’re welcome to change the name of your source files from the desktop. On Mac OS X and Windows, however, the desktop interface often hides file extensions, so check the properties of the file afterward to make sure you got the name you expected.

In a convenient location (e.g., your home directory or the desktop) create a new directory for your project called tutorial. In this new directory, save your DrRacket file as poem.html.pp.

"/path/to/tutorial/poem.html.pp"

#lang pollen

 

The margin is 42em.

The border is red.

The padding is 15em.

The border is too.

5.4 Using the project server

The project server is a web server built into Pollen. Just as DrRacket lets you run individual files and see if they work as you expect, the project server lets you preview and test your project as a real website. While working on your Pollen project, you may find it convenient to have DrRacket open on half your screen, and on the other half, a web browser pointing at the project server.

“Why can’t I just open the HTML files directly in my browser?” If you want to keep making web pages the way we did in 1996, go ahead. But that approach has several shortcomings. First, when you open files directly in your browser, you’re cruising the local filesystem, and absolute URLs (the kind that start with a /) won’t work. Second, if you want to test your website on devices other than your own machine — well, you can’t. Third, you have to render your HTML files in advance, whereas the project server is clever about doing this dynamically.

So use the project server.

A note about security. The project server isn’t intended for real-world use, but rather as a development tool. That said, once you start the project server, it’s an actual web server running on your machine, and it will respond to requests from any computer. If you want to limit traffic to your local network, or certain machines on your local network, it’s your job — not mine — to configure your firewall or other network security measures accordingly.

5.4.1 Starting the project server with raco pollen

Before we start the project server, a word about the raco pollen command.

When you installed Racket, Racket installed a utility program called raco. This name is short for Racket command, and raco acts as a hub for — you guessed it — Racket commands. You used it when you first installed Pollen:

> raco pkg install pollen

The first argument after raco is the subcommand. For instance, raco pkg ... lets you install, update, and remove packages like so:

> raco pkg update pollen

> raco pkg remove pollen

Likewise, raco pollen lets you issue commands relevant to Pollen, like starting the project server. (See Using raco pollen for a full description of available commands.)

Now we’ll start the project server. Go to your command line and enter the following:

> cd /path/to/tutorial

> raco pollen start

Windows users, I’ll trust you to convert raco into the appropriate command for your system — assuming defaults, it’s likely to be "C:\Program Files\Racket\raco" (include the surrounding quotes in the command).

After a moment, you’ll see a startup message like this:

Welcome to Pollen 0.001 (Racket 6.x.x.x)

Project root is /path/to/tutorial

Project server is http://localhost:8080 (Ctrl-C to exit)

Project dashboard is http://localhost:8080/index.ptree

Ready to rock

Project root means the directory that the project server was started in, and which it’s treating as its root directory. Any absolute URLs (i.e., those beginning with /) will resolve into this directory. So a URL like /styles.css will impliedly become /path/to/tutorial/styles.css.

If you use the bare command raco pollen start, the project server will start in the current directory. But if you want to start the project server elsewhere, you can add that directory as an argument like this:

> raco pollen start /some/other/path

The next line of the startup message tells you that the web address of the project server is http://localhost:8080. This is the address you put into your web browser to test your project. If you’re unfamiliar with this style of URL, localhost refers to your own machine, and 8080 is the network port where the project server will respond to browser requests.

If you want to access the project server from a different machine, you can’t use localhost. But you can use the IP address of the machine running the project server (e.g., http://192.168.1.10:8080) or any name for that machine available through local DNS (e.g., http://mb-laptop:8080).

Though port 8080 is the default, you can start the project server on any port you like by adding it as an argument to raco pollen start:

> raco pollen start /path/to/tutorial

> raco pollen start /path/to/tutorial 8088

You can also change the default port by altering world:default-port, or parameterizing it with world:current-server-port.

Note that when you pass a port argument, you also have to pass a path argument. If you want the project server to start in the current directory, you can use the usual . shorthand:

> cd /path/to/tutorial

> raco pollen start 8088

/path/to/tutorial/8088 is not a directory

> raco pollen start . 8088

You can run multiple project servers simultaneously. Just start them on different ports so they don’t conflict with each other.

Your terminal window will report status and error messages from the project server as it runs. Use Ctrl-C to stop the server.

5.4.2 Using the dashboard

For each directory in your project, starting at the top, the project server displays a dashboard in your web browser. The dashboard gives you an overview of the files in the directory, and links to view them.

The address of the top-level dashboard is http://localhost:8080/index.ptree. Other dashboards follow the same pattern (e.g., http://localhost:8080/path/to/dir/index.ptree.)

Note that the dashboard is not at http://localhost:8080/ or its equivalent, http://localhost:8080/index.html. Why? So it doesn’t interfere with any index.html that you may want to put in your project.

Thus, index.ptree. The .ptree extension is short for pagetree. In Pollen, a pagetree is a hierarchical list of pages. We’ll do more with pagetrees in a later tutorial. For now, just be aware that to generate the dashboard, the project server will first look for an actual index.ptree file in each directory. If it doesn’t find one, it will generate a pagetree from a listing of files in the directory.

Let’s look at the root-level dashboard for our project. First, make sure your project server is running:

> cd /path/to/tutorial

> raco pollen start

Then, in your web browser, visit http://localhost:8080/index.ptree.

You should see something like this:

The top line tells us that we’re in the root directory of the project. We didn’t make an explicit index.ptree file, so the project server just shows us a directory listing.

5.4.3 Source files in the dashboard

We see the only file, poem.html.pp. Note that the .pp extension is grayed out. The dashboard automatically consolidates references to source and output files into a single entry. What this entry says is “The directory contains a source file in .pp format for the output file poem.html.”

Every source-file entry in the dashboard has three links. The first link is attached to the filename itself, and takes you to a preview of the output file. If the output file doesn’t yet exist — as is the case here — it will be dynamically rendered. (This is true whether you click its name in the dashboard, or link to it from another page.) So click the filename. You’ll see in your web browser:

The margin is 42em. The border is red. The padding is 15em. The border is too.

Granted, this is a boring web page. The main point here is that you’re seeing the output from your source file, which didn’t exist before. Notice that the address bar says http://localhost:8080/poem.html, not poem.html.pp. And if you look in your tutorial directory, you’ll see a new file called poem.html.

In other words, when you clicked on the filename link in the dashboard, Pollen rendered the output file from your source file and saved it in your project directory. As promised earlier, the name of the output file (poem.html) is the name of the source file (poem.html.pp) minus the Pollen extension (.pp).

If you go back to the dashboard and click on the filename link again, you’ll see the same output file. If the source file hasn’t changed, Pollen will just show you the output file that’s already been rendered.

But if you like, open your poem.html.pp source file in DrRacket, edit the first line, and save the file:

#lang pollen

 

The cave is pitch black.

Look out for the grue.

The padding is 15em.

The border is too.

Go back to the dashboard and click on the filename. This time, you’ll see:

The cave is pitch black. Look out for the grue. The padding is 15em. The border is too.

Here, Pollen notices that the source file has changed, so it refreshes the output file. This makes it convenient to work between DrRacket and your web browser, editing source and then reloading to see the changes.

The other two links in the dashboard are labeled in and out.

The link labeled in will display the contents of the source file:

#lang pollen

 

The cave is pitch black.

Look out for the grue.

The padding is 15em.

The border is too.

The link labeled out will display the contents of the output file (just like the “view source” option in your web browser):

The cave is pitch black.

Look out for the grue.

The padding is 15em.

The border is too.

For now, the files are identical except for the #lang line. But let’s change that.

5.5 Working with the preprocessor

Pollen can operate in several processing modes. One of these is preprocessor mode. A preprocessor is a tool for making systematic, automated changes to a file, often in contemplation of further processing (hence the pre-). You can use the Pollen preprocessor this way. Or you can just use it on its own, and leave your files in a finished state.

That’s how we’ll use it in this tutorial. We’ll build out our poem.html.pp source file so that it exits the preprocessor as a legit HTML file.

5.5.1 Setting up a preprocessor source file

The file extension of a Pollen source file tells Pollen what kind of processing to apply to it. The “.pp” file extension stands for “Pollen preprocessor.” You can use the preprocessor with any text-based file by: +
The border is too.

This shows you something important: by default, any plain text in a Pollen source file is simply printed as written when you Run the file (minus the #lang line, which is just for Racket’s benefit). If you like, edit the text of the poem and click Run again. You’ll see the updated text printed in the interactions window.

5.3.3 Saving & naming your source file

File naming in Pollen is consequential.

Ultimately, every Pollen source file in your project will be rendered into an output file. Each Pollen source file corresponds to one output file. The name of this output file will be the name of the source file minus the Pollen source extension. So a source file called file.txt.pp will become file.txt.

Thus, to build the name of a source file, we take the name we want for the output file and add the appropriate Pollen file extension. Different Pollen source files use different extensions — but more about that later. For now, the extension you’ll use for your source is .pp.

In this case, let’s say we want to end up with a file called poem.html. Therefore, the name of our source file needs to be:

the output name poem.html + the source extension .pp = poem.html.pp

(If you want to name the file something-else.html.pp, be my guest. There’s no magic associated with the prefix.)

You’re welcome to change the name of your source files from the desktop. On Mac OS X and Windows, however, the desktop interface often hides file extensions, so check the properties of the file afterward to make sure you got the name you expected.

In a convenient location (e.g., your home directory or the desktop) create a new directory for your project called tutorial. In this new directory, save your DrRacket file as poem.html.pp.

"/path/to/tutorial/poem.html.pp"

#lang pollen

 

The margin is 42em.

The border is red.

The padding is 15em.

The border is too.

5.4 Using the project server

The project server is a web server built into Pollen. Just as DrRacket lets you run individual files and see if they work as you expect, the project server lets you preview and test your project as a real website. While working on your Pollen project, you may find it convenient to have DrRacket open on half your screen, and on the other half, a web browser pointing at the project server.

“Why can’t I just open the HTML files directly in my browser?” If you want to keep making web pages the way we did in 1996, go ahead. But that approach has several shortcomings. First, when you open files directly in your browser, you’re cruising the local filesystem, and absolute URLs (the kind that start with a /) won’t work. Second, if you want to test your website on devices other than your own machine — well, you can’t. Third, you have to render your HTML files in advance, whereas the project server is clever about doing this dynamically.

So use the project server.

A note about security. The project server isn’t intended for real-world use, but rather as a development tool. That said, once you start the project server, it’s an actual web server running on your machine, and it will respond to requests from any computer. If you want to limit traffic to your local network, or certain machines on your local network, it’s your job — not mine — to configure your firewall or other network security measures accordingly.

5.4.1 Starting the project server with raco pollen

Before we start the project server, a word about the raco pollen command.

When you installed Racket, Racket installed a utility program called raco. This name is short for Racket command, and raco acts as a hub for — you guessed it — Racket commands. You used it when you first installed Pollen:

> raco pkg install pollen

The first argument after raco is the subcommand. For instance, raco pkg ... lets you install, update, and remove packages like so:

> raco pkg update pollen

> raco pkg remove pollen

Likewise, raco pollen lets you issue commands relevant to Pollen, like starting the project server. (See Using raco pollen for a full description of available commands.)

Now we’ll start the project server. Go to your command line and enter the following:

> cd /path/to/tutorial

> raco pollen start

Windows users, I’ll trust you to convert raco into the appropriate command for your system — assuming defaults, it’s likely to be "C:\Program Files\Racket\raco" (include the surrounding quotes in the command).

After a moment, you’ll see a startup message like this:

Welcome to Pollen 0.001 (Racket 6.x.x.x)

Project root is /path/to/tutorial

Project server is http://localhost:8080 (Ctrl-C to exit)

Project dashboard is http://localhost:8080/index.ptree

Ready to rock

Project root means the directory that the project server was started in, and which it’s treating as its root directory. Any absolute URLs (i.e., those beginning with /) will resolve into this directory. So a URL like /styles.css will impliedly become /path/to/tutorial/styles.css.

If you use the bare command raco pollen start, the project server will start in the current directory. But if you want to start the project server elsewhere, you can add that directory as an argument like this:

> raco pollen start /some/other/path

The next line of the startup message tells you that the web address of the project server is http://localhost:8080. This is the address you put into your web browser to test your project. If you’re unfamiliar with this style of URL, localhost refers to your own machine, and 8080 is the network port where the project server will respond to browser requests.

If you want to access the project server from a different machine, you can’t use localhost. But you can use the IP address of the machine running the project server (e.g., http://192.168.1.10:8080) or any name for that machine available through local DNS (e.g., http://mb-laptop:8080).

Though port 8080 is the default, you can start the project server on any port you like by adding it as an argument to raco pollen start:

> raco pollen start /path/to/tutorial

> raco pollen start /path/to/tutorial 8088

You can also change the default port by altering world:default-port, or parameterizing it with world:current-server-port.

Note that when you pass a port argument, you also have to pass a path argument. If you want the project server to start in the current directory, you can use the usual . shorthand:

> cd /path/to/tutorial

> raco pollen start 8088

/path/to/tutorial/8088 is not a directory

> raco pollen start . 8088

You can run multiple project servers simultaneously. Just start them on different ports so they don’t conflict with each other.

Your terminal window will report status and error messages from the project server as it runs. Use Ctrl-C to stop the server.

5.4.2 Using the dashboard

For each directory in your project, starting at the top, the project server displays a dashboard in your web browser. The dashboard gives you an overview of the files in the directory, and links to view them.

The address of the top-level dashboard is http://localhost:8080/index.ptree. Other dashboards follow the same pattern (e.g., http://localhost:8080/path/to/dir/index.ptree.)

Note that the dashboard is not at http://localhost:8080/ or its equivalent, http://localhost:8080/index.html. Why? So it doesn’t interfere with any index.html that you may want to put in your project.

Thus, index.ptree. The .ptree extension is short for pagetree. In Pollen, a pagetree is a hierarchical list of pages. We’ll do more with pagetrees in a later tutorial. For now, just be aware that to generate the dashboard, the project server will first look for an actual index.ptree file in each directory. If it doesn’t find one, it will generate a pagetree from a listing of files in the directory.

Let’s look at the root-level dashboard for our project. First, make sure your project server is running:

> cd /path/to/tutorial

> raco pollen start

Then, in your web browser, visit http://localhost:8080/index.ptree.

You should see something like this:

The top line tells us that we’re in the root directory of the project. We didn’t make an explicit index.ptree file, so the project server just shows us a directory listing.

5.4.3 Source files in the dashboard

We see the only file, poem.html.pp. Note that the .pp extension is grayed out. The dashboard automatically consolidates references to source and output files into a single entry. What this entry says is “The directory contains a source file in .pp format for the output file poem.html.”

Every source-file entry in the dashboard has three links. The first link is attached to the filename itself, and takes you to a preview of the output file. If the output file doesn’t yet exist — as is the case here — it will be dynamically rendered. (This is true whether you click its name in the dashboard, or link to it from another page.) So click the filename. You’ll see in your web browser:

The margin is 42em. The border is red. The padding is 15em. The border is too.

Granted, this is a boring web page. The main point here is that you’re seeing the output from your source file, which didn’t exist before. Notice that the address bar says http://localhost:8080/poem.html, not poem.html.pp. And if you look in your tutorial directory, you’ll see a new file called poem.html.

In other words, when you clicked on the filename link in the dashboard, Pollen rendered the output file from your source file and saved it in your project directory. As promised earlier, the name of the output file (poem.html) is the name of the source file (poem.html.pp) minus the Pollen extension (.pp).

If you go back to the dashboard and click on the filename link again, you’ll see the same output file. If the source file hasn’t changed, Pollen will just show you the output file that’s already been rendered.

But if you like, open your poem.html.pp source file in DrRacket, edit the first two lines, and save the file:

#lang pollen

 

The cave is pitch black.

Look out for the grue.

The padding is 15em.

The border is too.

Go back to the dashboard and click on the filename. This time, you’ll see:

The cave is pitch black. Look out for the grue. The padding is 15em. The border is too.

Here, Pollen notices that the source file has changed, so it refreshes the output file. This makes it convenient to work between DrRacket and your web browser, editing source and then reloading to see the changes.

The other two links in the dashboard are labeled in and out.

The link labeled in will display the contents of the source file:

#lang pollen

 

The cave is pitch black.

Look out for the grue.

The padding is 15em.

The border is too.

The link labeled out will display the contents of the output file (just like the “view source” option in your web browser):

The cave is pitch black.

Look out for the grue.

The padding is 15em.

The border is too.

For now, the files are identical except for the #lang line. But let’s change that.

5.5 Working with the preprocessor

Pollen can operate in several processing modes. One of these is preprocessor mode. A preprocessor is a tool for making systematic, automated changes to a file, often in contemplation of further processing (hence the pre-). You can use the Pollen preprocessor this way. Or you can just use it on its own, and leave your files in a finished state.

That’s how we’ll use it in this tutorial. We’ll build out our poem.html.pp source file so that it exits the preprocessor as a legit HTML file.

5.5.1 Setting up a preprocessor source file

The file extension of a Pollen source file tells Pollen what kind of processing to apply to it. The “.pp” file extension stands for “Pollen preprocessor.” You can use the preprocessor with any text-based file by:

  • inserting #lang pollen as the first line,

  • adding the .pp file extension,

  • running it through Pollen.

For more about the Pollen processing modes and how to invoke them, see File.

“The preprocessor be used with any kind of text-based file?” Right. “But how?” The preprocessor reads the source file, handles any Pollen commands it finds, and lets the rest of the content pass through untouched. To the preprocessor, it’s all just text data. It doesn’t care whether that text represents HTML, CSS, JavaScript, or even TI-BASIC.

Because the preprocessor only deals in text, the Pollen commands you use in the preprocessor also have to produce text. Moreover, Pollen doesn’t enforce the semantics of the underlying file — that’s your responsibility. For instance, Pollen won’t stop you from doing nonsensical things like this:

"bad-poem.html.pp"

#lang pollen

 

The cave is pitch black.

Look out for the grue.

◊(insert-mp3-recording-of-scream)

Here, the result is not going to be valid HTML, because you can’t simply drop binary data in the middle of an HTML file. To paraphrase Mr. Babbage — garbage in, garbage out.

I’ve encouraged you to mess with the source file, but let’s return it to its original state:

"/path/to/tutorial/poem.html.pp"

#lang pollen

 

The margin is 42em.

The border is red.

The padding is 15em.

The border is too.

This file has #lang pollen as the first line, and .pp as the file extension, so it meets the minimum requirements for the preprocessor.

5.5.2 Creating valid HTML output

Let’s update our source so it produces valid HTML. Edit the source as follows:

"/path/to/tutorial/poem.html.pp"

#lang pollen

<!DOCTYPE html>

<html>

<body>

<pre>

The margin is 42em.

The border is red.

The padding is 15em.

The border is too.

</pre>

</body>

</html>

Return to the project server and view http://localhost:8080/poem.html. Earlier, the output looked like this:

The margin is 42em. The border is red. The padding is 15em. The border is too.

But now, because of the <pre> tag, the poem will appear in a monospaced font, and the line breaks will be preserved:

The margin is 42em.
The border is red.
The padding is 15em. diff --git a/scribblings/decode.scrbl b/scribblings/decode.scrbl index 09fcff8..57f1d7f 100644 --- a/scribblings/decode.scrbl +++ b/scribblings/decode.scrbl @@ -45,7 +45,15 @@ Recall from [future link: Pollen mechanics] that any tag can have a function att For instance, here's how @racket[decode] is attached to @racket[root] in @italic{Butterick's Practical Typography}. There's not much to it — -[update with actual code] +@racketblock[ +(define (root . items) + (decode (make-txexpr 'root '() items) + #:txexpr-elements-proc detect-paragraphs + #:block-txexpr-proc (compose1 hyphenate wrap-hanging-quotes + insert-nonbreaking-last-space) + #:string-proc (compose1 smart-quotes smart-dashes) + #:exclude-tags '(style script))) + ] This illustrates another important point: even though @racket[decode] presents an imposing list of arguments, you're unlikely to use all of them at once. These represent possibilities, not requirements. For instance, let's see what happens when @racket[decode] is invoked without any of its optional arguments.