unbound identifier error #233

I have been using pollen, almost daily, for a year now -- thank you for this simple and versatile tool.

This morning, raco pollen render yielded an unbound identifier error. (The same command on the same file worked the day before.)

This looked similar to #28, so I updated, including dependencies (raco pkg update --update-deps pollen). However, the problem remains. I have pasted the entire error message below, but please do let me know if any further information might help.

Thank you in advance for any idea you might have on what is going wrong.

[Linux, Racket 7.8-1]

> raco pollen render -t ltx ./filename.poly.pm
pollen: rendering ./filename.poly.pm
markup: unbound identifier
  in: markup
  context...:
   do-raise-syntax-error
   for-loop
   [repeats 2 more times]
   finish-bodys
   lambda-clause-expander
   for-loop
   loop
   [repeats 6 more times]
   module-begin-k
   expand-module
   expand-capturing-lifts
   temp98_0
   temp71_0
   compile
   /usr/share/racket/collects/compiler/private/cm-minimal.rkt:608:0: compile-zo*
   /usr/share/racket/collects/compiler/private/cm-minimal.rkt:407:15
   ...

I have been using pollen, almost daily, for a year now -- thank you for this simple and versatile tool. This morning, `raco pollen render` yielded an `unbound identifier` error. (The same command on the same file worked the day before.) This looked similar to #28, so I updated, including dependencies (`raco pkg update --update-deps pollen`). However, the problem remains. I have pasted the entire error message below, but please do let me know if any further information might help. Thank you in advance for any idea you might have on what is going wrong. [Linux, Racket 7.8-1] ~~~ > raco pollen render -t ltx ./filename.poly.pm pollen: rendering ./filename.poly.pm markup: unbound identifier in: markup context...: do-raise-syntax-error for-loop [repeats 2 more times] finish-bodys lambda-clause-expander for-loop loop [repeats 6 more times] module-begin-k expand-module expand-capturing-lifts temp98_0 temp71_0 compile /usr/share/racket/collects/compiler/private/cm-minimal.rkt:608:0: compile-zo* /usr/share/racket/collects/compiler/private/cm-minimal.rkt:407:15 ... ~~~

Is it still working for other .poly.pm files in the same project?

Do you have a setup module in your pollen.rkt? Have you changed it lately?

Is it still working for other `.poly.pm` files in the same project? Do you have a `setup` module in your `pollen.rkt`? Have you changed it lately?

Thanks for the swift response.

I don't have other .poly.pm files in the same project, but the same thing happened for another .poly.pm file in another directory.

My setup module is as follows:

(module setup racket/base
  (provide (all-defined-out))
  (define poly-targets '(md html txt ltx)))

Thanks for the swift response. I don't have other `.poly.pm` files in the same project, but the same thing happened for another `.poly.pm` file in another directory. My setup module is as follows: ~~~ (module setup racket/base (provide (all-defined-out)) (define poly-targets '(md html txt ltx))) ~~~

Puzzling! Can you create a new, minimal .poly.pm file in the same directory and reproduce the error when rendering it? Or would you mind posting the original .poly.pm file?

Puzzling! Can you create a new, minimal `.poly.pm` file in the same directory and reproduce the error when rendering it? Or would you mind posting the original `.poly.pm` file?

Thanks again. I just got the same error with a file named test.poly.pm in the same folder. Here are its contents:

#lang pollen

◊(define-meta title "Test")
◊(define-meta author "test")
◊(define-meta course_duration "2020")

◊chapter{Test}

Test test test

Thanks again. I just got the same error with a file named `test.poly.pm` in the same folder. Here are its contents: ~~~ #lang pollen ◊(define-meta title "Test") ◊(define-meta author "test") ◊(define-meta course_duration "2020") ◊chapter{Test} Test test test ~~~

Can you point us to the repo that hosts your code?

Unfortunately, it is not public (and would take a lot of cleaning up before being ready).

However, it looks like the backtick in the following line in my pollen.rkt was the culprit. Removing this line solved the issue.

(define (quote text) (format "`~a\'" text))

(On the other hand, escaping the backtick with a backslash yields:)

pollen.rkt:86:29: read-syntax: unknown escape sequence `\`` in string

Apologies if I am simply revealing my ignorance of a facet of Racket here. Thanks again to both of you.

Unfortunately, it is not public (and would take a lot of cleaning up before being ready). However, it looks like the backtick in the following line in my `pollen.rkt` was the culprit. Removing this line solved the issue. ~~~ (define (quote text) (format "`~a\'" text)) ~~~ (On the other hand, escaping the backtick with a backslash yields:) ~~~ pollen.rkt:86:29: read-syntax: unknown escape sequence `\`` in string ~~~ Apologies if I am simply revealing my ignorance of a facet of Racket here. Thanks again to both of you.

OK I can reproduce it now. This is interesting. It’s not the backtick, but rather the name of your function: quote is the name of a very basic form in Racket. As a meta-language tool, Racket will happily let you re-define (“shadow”) any of its built-in functions and macros, but you will get unexpected behavior if you don’t fully understand what you’re doing.

As explained in the linked docs, when Racket sees 'symbol it interprets it as (quote symbol) which would in turn normally use the quote macro from racket/base. But now it would use the quote from your pollen.rkt. (More on this in a bit)

Two reasons this is interesting:

I’m actually kind of surprised this hasn’t come up before now, since when designing markup it makes complete sense to use the name quote to quote some text.
Given this, I’m now curious how your pages ever worked! Did you add this function recently?

OK I can reproduce it now. This is interesting. It’s not the backtick, but rather the name of your function: `quote` is the name of [a very basic form in Racket](https://docs.racket-lang.org/guide/quote.html). As a meta-language tool, Racket will happily let you re-define (“shadow”) any of its built-in functions and macros, but you will get unexpected behavior if you don’t fully understand what you’re doing. As explained in the linked docs, when Racket sees `'symbol` it interprets it as `(quote symbol)` which would in turn normally use the `quote` macro from `racket/base`. But now it would use the `quote` from your `pollen.rkt`. (More on this in a bit) Two reasons this is interesting: 1. I’m actually kind of surprised this hasn’t come up before now, since when designing markup it makes complete sense to use the name `quote` to quote some text. 2. Given this, I’m now curious how your pages *ever* worked! Did you add this function recently?

More technical info: This is just me trying to figure out why you go the particular error you did.

Behind the scenes, when your .poly.pm file is run, it first goes through an expander phase, where any macro forms are replaced with the results of running the macro functions on their contents.

If you open a .poly.pm file and click the Macro Stepper you’ll see what this particular program (your documents are programs after all) looks like to Racket after it has expanded Pollen’s macros but before expanding any macros local to your file:

(module runtime-wrapper racket/base
  (module configure-runtime racket/base
    (require pollen/private/runtime-config)
    (configure "/Users/joel/Documents/code/sandbox/test.html.pm"))
  (module pollen-module pollen/private/main-base
    'markup
     ; [your markup and metas are in here…]
  (module metas racket/base
    (require (submod ".." pollen-module metas))
    (provide metas))
  ; […]

If you click the Step button, you’ll see expansion goes fine until it gets to the metas submodule which in turn requires the pollen-module submodule. This evidently forces Racket to fully evaluate the code in that module, at whih point it encounters 'markup. Racket interprets this as (quote markup) which would normally just leave a symbol there that the code in pollen/private/main-base will look for to know what “mode” this file is (i.e. markdown, markup, pagetree, etc).

But apparently, already at this stage the quote function defined in your pollen.rkt holds sway, and when Racket evaluates (quote markup) it now expects markup to be a bound identifier just as it would in any other function call. Because it’s not, it fails and you get the error.

What I’m unclear on is why the definitions in pollen.rkt matter at this stage. 'markup is inserted by the Pollen reader here but any bindings there are stripped out by the reader, so quote can’t be bound to the pollen.rkt version at that point. The expander for the module is pollen-module-begin but I’m not sure quite how bindings from pollen.rkt are affecting the 'markup at the beginning of that macro.

**More technical info:** This is just me trying to figure out why you go the particular error you did. Behind the scenes, when your `.poly.pm` file is run, it first goes through an *expander* phase, where any macro forms are replaced with the results of running the macro functions on their contents. If you open a `.poly.pm` file and click the _Macro Stepper_ you’ll see what this particular program (your documents are programs after all) looks like to Racket after it has expanded Pollen’s macros but before expanding any macros local to your file: ```racket (module runtime-wrapper racket/base (module configure-runtime racket/base (require pollen/private/runtime-config) (configure "/Users/joel/Documents/code/sandbox/test.html.pm")) (module pollen-module pollen/private/main-base 'markup ; [your markup and metas are in here…] (module metas racket/base (require (submod ".." pollen-module metas)) (provide metas)) ; […] ``` If you click the _Step_ button, you’ll see expansion goes fine until it gets to the `metas` submodule which in turn `require`s the `pollen-module` submodule. This evidently forces Racket to fully evaluate the code in that module, at whih point it encounters `'markup`. Racket interprets this as `(quote markup)` which would normally just leave a symbol there that the code in `pollen/private/main-base` will look for to know what “mode” this file is (i.e. markdown, markup, pagetree, etc). But apparently, already at this stage the `quote` function defined in your `pollen.rkt` holds sway, and when Racket evaluates `(quote markup)` it now expects `markup` to be a bound identifier just as it would in any other function call. Because it’s not, it fails and you get the error. What I’m unclear on is _why_ the definitions in `pollen.rkt` matter at this stage. `'markup` is inserted by the Pollen reader [here][1] but any bindings there are stripped out by the reader, so `quote` can’t be bound to the `pollen.rkt` version at that point. The expander for the module is [`pollen-module-begin`](https://github.com/mbutterick/pollen/blob/aa7aa7a2e433ee06b80d8ed37c744e959c2bb6cc/pollen/private/main-base.rkt#L28) but I’m not sure quite how bindings from `pollen.rkt` are affecting the `'markup` at the beginning of that macro. [1]: https://github.com/mbutterick/pollen/blob/aa7aa7a2e433ee06b80d8ed37c744e959c2bb6cc/pollen/private/reader-base.rkt#L65

A technique called partial expansion ensures that all forms in the parse tree that might introduce bindings (like require) have been discovered before expanding the other forms — because of course, those bindings will affect how those other forms should be expanded. Matthew Flatt describes it here.

In this case, even though the quote binding in "pollen.rkt" appears later in the source, it gets handled during the partial-expansion pass. Then Racket, now wielding the unintentionally redefined quote, goes back and evalutes the 'markup expression at the top, which has been read into the parse tree as (quote markup).

A technique called [partial expansion](https://docs.racket-lang.org/reference/syntax-model.html#%28part._partial-expansion%29) ensures that all forms in the parse tree that might introduce bindings (like `require`) have been discovered before expanding the other forms — because of course, those bindings will affect how those other forms should be expanded. Matthew Flatt [describes it here](https://groups.google.com/d/msg/racket-users/PRtcLBO01QU/yvgIlB4JDgAJ). In this case, even though the `quote` binding in `"pollen.rkt"` appears later in the source, it gets handled during the partial-expansion pass. Then Racket, now wielding the unintentionally redefined `quote`, goes back and evalutes the `'markup` expression at the top, which has been read into the parse tree as `(quote markup)`.

Thank you both for these (very interesting) explanations. I learned Racket in order to use Pollen (and chose Pollen partially because I wanted to explore a Lisp), so "if you don’t fully understand what you’re doing" certainly applies.

To answer your question, Joel, (I now realise that) this only cropped up at this point because I just renamed the function from inline-quotation to quote. I didn't realise that that was the problem because in the very same edit I introduced the backticks, having used apostrophes till then.

I appreciate the detail of your explanation -- in future I'll try to avoid such 'reserved words'.

Thank you both for these (very interesting) explanations. I learned Racket in order to use Pollen (and chose Pollen partially because I wanted to explore a Lisp), so "if you don’t fully understand what you’re doing" certainly applies. To answer your question, Joel, (I now realise that) this only cropped up at this point because I just renamed the function from `inline-quotation` to `quote`. I didn't realise that that was the problem because in the very same edit I introduced the backticks, having used apostrophes till then. I appreciate the detail of your explanation -- in future I'll try to avoid such 'reserved words'.

Labels Milestones

unbound identifier error #233