Should Pollen templates be given a #lang? #86

I am going through the Pollen tutorials and am finding them well written and enjoyable, but figured I'd comment an impression I had: Pollen templates feel a bit like second class citizens.

Disclaimer: I am both a Racket and a Pollen newbie, though I have a lot of "lispy" experience in other systems (Emacs, Common Lisp).

Templates don't have a #lang. Yet, when editing them, I am still programming -- but in what language? I wonder what benefits could be had if Pollen had something like:

#lang pollen-template

This question strikes me as a surprising one to have of a Racket system, where there seems to be a strong focus on identifying the semantics of hand written program text, expressing those semantics in a #lang, and making each #lang a good programming experience by, among other benefits, providing helpful "IDE like" features, good error messages, etc.

Two situation from Tutorial 2:

When editing a template the #lang pollen line is gone, so the "Insert command char ◊" in DrRacket disappears. DrRacket also issues warnings that the "<html" is an invalid module (or something like that). This is confusing for the new Racket user (me).
Later, there is an exception thrown when the template has some bad code in it. This exception, arguably, is a "bad" error -- i.e. the line/colum of the offending pollen code, relative to the template file itself, isn't part of the error message. I wonder: could that error be improved upon if a template were a full blown Racket #lang?

In other publishing systems I have usually found editing the templates to be the most confusing and time consuming things. This is why I was looking at Pollen's treatment of templates closely.

#77 seems like related question.

I am going through the Pollen tutorials and am finding them well written and enjoyable, but figured I'd comment an impression I had: Pollen templates feel a bit like second class citizens. Disclaimer: I am both a Racket and a Pollen newbie, though I have a lot of "lispy" experience in other systems (Emacs, Common Lisp). Templates don't have a `#lang`. Yet, when editing them, I am still programming -- but in what language? I wonder what benefits could be had if Pollen had something like: ``` #lang pollen-template ``` This question strikes me as a surprising one to have of a Racket system, where there seems to be a strong focus on identifying the semantics of hand written program text, expressing those semantics in a `#lang`, and making each `#lang` a good programming experience by, among other benefits, providing helpful "IDE like" features, good error messages, etc. Two situation from Tutorial 2: 1. When editing a template the `#lang pollen` line is gone, so the "Insert command char ◊" in DrRacket disappears. DrRacket also issues warnings that the "<html" is an invalid module (or something like that). This is confusing for the new Racket user (me). 2. Later, there is an exception thrown when the template has some bad code in it. This exception, arguably, is a "bad" error -- i.e. the line/colum of the offending pollen code, relative to the template file itself, isn't part of the error message. I wonder: could that error be improved upon if a template were a full blown Racket `#lang`? In other publishing systems I have usually found editing the templates to be the most confusing and time consuming things. This is why I was looking at Pollen's treatment of templates closely. #77 seems like related question.

One thing could be that one doesn’t want to limit himself only to html templates, or markdown templates. In theory, you can template any text-based file with pollen at the moment, and introducing a specialised #lang html-template would hinder this. Not to say that it would be bad to have one (I guess 95% of the time you are indeed templating html or css), but it might shine some light on the though process behind this.

We could probably make a lang that would just ignore the non-pollen bits in the template, but that would also be a unideal, because fot those bits (html/css/…) you want things like auto-completion and proper syntax highlighting. It really is hard to have the best of both worlds.

1. 2021 v 20:37, Matt Armstrong notifications@github.com:

I am going through the Pollen tutorials and am finding them well written and enjoyable, but figured I'd comment an impression I had: Pollen templates feel a bit like second class citizens.

Disclaimer: I am both a Racket and a Pollen newbie, though I have a lot of "lispy" experience in other systems (Emacs, Common Lisp).

Templates don't have a #lang. Yet, when editing them, I am still programming -- but in what language? I wonder what benefits could be had if Pollen had something like:

#lang pollen-template
This question strikes me as a surprising one to have of a Racket system, where there seems to be a strong focus on identifying the semantics of hand written program text, expressing those semantics in a #lang, and making each #lang a good programming experience by, among other benefits, providing helpful "IDE like" features, good error messages, etc.

Two situation from Tutorial 2:

When editing a template the #lang pollen line is gone, so the "Insert command char ◊" in DrRacket disappears. DrRacket also issues warnings that the "<html" is an invalid module (or something like that). This is confusing for the new Racket user (me).
Later, there is an exception thrown when the template has some bad code in it. This exception, arguably, is a "bad" error -- i.e. the line/colum of the offending pollen code, relative to the template file itself, isn't part of the error message. I wonder: could that error be improved upon if a template were a full blown Racket #lang?
#77 https://github.com/mbutterick/pollen-users/issues/77 seems like related question.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/mbutterick/pollen-users/issues/86, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFCF3IHSJ6WSXLXHHMOVJN3S5GQYZANCNFSM4XBOL3UQ.

One thing could be that one doesn’t want to limit himself only to html templates, or markdown templates. In theory, you can template any text-based file with pollen at the moment, and introducing a specialised #lang html-template would hinder this. Not to say that it would be bad to have one (I guess 95% of the time you are indeed templating html or css), but it might shine some light on the though process behind this. We could probably make a lang that would just ignore the non-pollen bits in the template, but that would also be a unideal, because fot those bits (html/css/…) you want things like auto-completion and proper syntax highlighting. It really is hard to have the best of both worlds. 3. 2. 2021 v 20:37, Matt Armstrong <notifications@github.com>: I am going through the Pollen tutorials and am finding them well written and enjoyable, but figured I'd comment an impression I had: Pollen templates feel a bit like second class citizens. Disclaimer: I am both a Racket and a Pollen newbie, though I have a lot of "lispy" experience in other systems (Emacs, Common Lisp). Templates don't have a #lang. Yet, when editing them, I am still programming -- but in what language? I wonder what benefits could be had if Pollen had something like: #lang pollen-template This question strikes me as a surprising one to have of a Racket system, where there seems to be a strong focus on identifying the semantics of hand written program text, expressing those semantics in a #lang, and making each #lang a good programming experience by, among other benefits, providing helpful "IDE like" features, good error messages, etc. Two situation from Tutorial 2: When editing a template the #lang pollen line is gone, so the "Insert command char ◊" in DrRacket disappears. DrRacket also issues warnings that the "<html" is an invalid module (or something like that). This is confusing for the new Racket user (me). Later, there is an exception thrown when the template has some bad code in it. This exception, arguably, is a "bad" error -- i.e. the line/colum of the offending pollen code, relative to the template file itself, isn't part of the error message. I wonder: could that error be improved upon if a template were a full blown Racket #lang? #77 <https://github.com/mbutterick/pollen-users/issues/77> seems like related question. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <https://github.com/mbutterick/pollen-users/issues/86>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFCF3IHSJ6WSXLXHHMOVJN3S5GQYZANCNFSM4XBOL3UQ>.

I have the same reservations about templates. But I’ve never had a better idea. The problem is that a template is not, by nature, a self-contained source file. It’s a fragment that’s only meaningful when combined with a lexical context from elsewhere. (BTW I didn’t invent that idea. Pollen’s templates are a thin layer around Racket’s web templates.)

Of course, templates are totally optional in Pollen — you could inject the same information into a page using tag functions, or root, etc. But it’s a workflow that’s convenient & familiar to people who have used other web-publishing systems.

I have the same reservations about templates. But I’ve never had a better idea. The problem is that a template is not, by nature, a self-contained source file. It’s a fragment that’s only meaningful when combined with a lexical context from elsewhere. (BTW I didn’t invent that idea. Pollen’s templates are a thin layer around Racket’s [web templates](https://docs.racket-lang.org/web-server/templates.html?q=templates).) Of course, templates are totally optional in Pollen — you could inject the same information into a page using tag functions, or `root`, etc. But it’s a workflow that’s convenient & familiar to people who have used other web-publishing systems.

I’ve used both approaches, having templates for some files and in other cases a .rkt file that grabs docs/metas it needs and writes out files manually. I’m usually loath to give up the conveniences of raco pollen render, though.

It might be possible to have the best of both worlds by allowing you to use, instead of a template, a module that must provide a single function, e.g. (render doc metas) with a signature like (-> any/c hash? (or/c string? bytes?)). Kind of like how when interpreting a #lang line Racket looks for a read-syntax function with a particular signature in a particular location.

I’ve used both approaches, having templates for some files and in other cases a `.rkt` file that grabs docs/metas it needs and writes out files manually. I’m usually loath to give up the conveniences of `raco pollen render`, though. It might be possible to have the best of both worlds by allowing you to use, instead of a template, a module that must `provide` a single function, e.g. `(render doc metas)` with a signature like `(-> any/c hash? (or/c string? bytes?))`. Kind of like how when interpreting a `#lang` line Racket looks for a `read-syntax` function with a particular signature in a particular location.

Then, too, if that were feasible, we could just as well have a #lang pollen/template using the Pollen reader that automatically surrounded its contents inside the definition of the necessary render function, and provided that function. I suppose it would have to watch out for require and other top-level forms.

Then, too, if that were feasible, we could just as well have a `#lang pollen/template` using the Pollen reader that automatically surrounded its contents inside the definition of the necessary `render` function, and provided that function. I suppose it would have to watch out for `require` and other top-level forms.

OK I’m taking a stab at this: https://github.com/otherjoel/beeswax

Doesn’t work yet, hopefully I’ll have more free time to finish it before too long.

OK I’m taking a stab at this: https://github.com/otherjoel/beeswax Doesn’t work yet, hopefully I’ll have more free time to finish it before too long.

@otherjoel that seems like an interesting idea though I can't say I fully understand it yet. ;-)

I have an example template lang working to the point where I can test it out in a non-trivial Pollen project: https://github.com/otherjoel/beeswax/issues/7

@mbutterick I would be interested in your thoughts! Your contributing guidelines suggest you’d prefer this to remain a separate package, which would be fine. The downside to me would be the work to duplicate the parallel rendering capabilities of raco pollen render.

I have an example template lang working to the point where I can test it out in a non-trivial Pollen project: https://github.com/otherjoel/beeswax/issues/7 @mbutterick I would be interested in your thoughts! Your contributing guidelines suggest you’d prefer this to remain a separate package, which would be fine. The downside to me would be the work to duplicate the parallel rendering capabilities of `raco pollen render`.

Pollen templates can refer to any identifiers that are in the lexical scope of the underlying Pollen source. So doc and metas, but also everything exported by the associated pollen.rkt. It sounds like beeswax trades some of this flexibility for speed (in the sense that it only handles doc and metas) and better usability (in the sense that a beeswax source can be compiled, whereas a Pollen template cannot). Is that right?

Pollen templates can refer to any identifiers that are in the lexical scope of the underlying Pollen source. So `doc` and `metas`, but also everything exported by the associated `pollen.rkt`. It sounds like `beeswax` trades some of this flexibility for speed (in the sense that it only handles `doc` and `metas`) and better usability (in the sense that a `beeswax` source can be compiled, whereas a Pollen template cannot). Is that right?

Leaving out pollen.rkt is definitely not a design choice; I just haven’t gotten to it yet. Ideally changing a Pollen template to this style of template would require no more work than adding the #lang line at the top.

If I continue to develop this as a separate project, I would definitely auto-require pollen.rkt and pollen/core in addition to the other Pollen modules I’m currently auto-requiring (pollen/pagetree and pollen/template). I’d also use the local setup:command-char for the reader. Note that beeswax also provides the here variable in templates as well.

Leaving out `pollen.rkt` is definitely not a design choice; I just haven’t gotten to it yet. Ideally changing a Pollen template to this style of template would require no more work than adding the `#lang` line at the top. If I continue to develop this as a separate project, I would definitely auto-require `pollen.rkt` and `pollen/core` in addition to the other Pollen modules I’m currently auto-requiring (`pollen/pagetree` and `pollen/template`). I’d also use the local `setup:command-char` for the reader. Note that `beeswax` also provides the `here` variable in templates as well.

prefer this to remain a separate package

Yes, though —

I’m open to improving the public interface of Pollen itself so you can extend it with beeswax (you would have to tell me what’s necessary)
You can use the pollen module namespace if you want.

> prefer this to remain a separate package Yes, though — 1. I’m open to improving the public interface of Pollen itself so you can extend it with `beeswax` (you would have to tell me what’s necessary) 2. You can use the `pollen` module namespace if you want.

I’m open to improving the public interface of Pollen itself so you can extend it

OK, I’ll keep that in mind!

You can use the pollen module namespace if you want.

That’s good to know as well. Although I’ll have to think about whether putting it in its own namespace would be less confusing for the typical Pollen user.

> I’m open to improving the public interface of Pollen itself so you can extend it OK, I’ll keep that in mind! > You can use the pollen module namespace if you want. That’s good to know as well. Although I’ll have to think about whether putting it in its own namespace would be less confusing for the typical Pollen user.

I’m open to improving the public interface of Pollen itself so you can extend it with beeswax (you would have to tell me what’s necessary)

It would be nice to have a Pollen-provided way of determining the path to the pollen.rkt file that applies to a particular Pollen source file. I could write my own (i.e., basically copy get-path-to-override from pollen/setup) but it seems more future-proof and less redundant to ask Pollen.

(Let me know if I should continue to post these here since I might be getting kind of off-topic.)

> I’m open to improving the public interface of Pollen itself so you can extend it with beeswax (you would have to tell me what’s necessary) It would be nice to have a Pollen-provided way of determining the path to the `pollen.rkt` file that applies to a particular Pollen source file. I could write my own (i.e., basically copy [`get-path-to-override`][1] from `pollen/setup`) but it seems more future-proof and less redundant to ask Pollen. (Let me know if I should continue to post these here since I might be getting kind of off-topic.) [1]: https://github.com/mbutterick/pollen/blob/1210e1b31d40d198b18b8a33d24eb5f4e3ad58f2/pollen/setup.rkt#L22-L33

Update — the still-experimental #lang for templates is now ready for public consumption: https://docs.racket-lang.org/beeswax/index.html ...tire-kicking welcome. (Questions/bugs should go to the repo)

Matthew—thanks again for accommodating this project within Pollen’s public interface!

Update — the still-experimental #lang for templates is now ready for public consumption: https://docs.racket-lang.org/beeswax/index.html ...tire-kicking welcome. (Questions/bugs should go to [the repo](https://github.com/otherjoel/beeswax)) Matthew—thanks again for accommodating this project within Pollen’s public interface!

Labels Milestones

Should Pollen templates be given a #lang? #86