Unexpected render-pagetree behaviour #78

The behaviour of render-pagetree is unexpected.

For example, running the following in a REPL...

(render-pagetree '(pagetree-root (index.html test.html)))

... given the .pm and template below should show the following:

index.html(index.html)(test.html)

but instead gives

index.html(index.html pollen.cache simple.html test.html)#f

i.e. shows all files in the current directory for (display (siblings here)) and #f for (display (children here))

#lang pollen/markup

◊meta['template: "simple.html"]
◊meta['title: "Index"]

<html>
  <head>
  </head>
  <body>
    ◊(display here)
    ◊(display (siblings here))
    ◊(display (children here))
    </body>
</html>

Seems like the template is seeing an autogenerated pagetree rather than the one I'm supplying.

The behaviour of `render-pagetree` is unexpected. For example, running the following in a REPL... ``` (render-pagetree '(pagetree-root (index.html test.html))) ``` ... given the .pm and template below should show the following: ``` index.html(index.html)(test.html) ``` but instead gives ``` index.html(index.html pollen.cache simple.html test.html)#f ``` i.e. shows all files in the current directory for `(display (siblings here))` and `#f` for `(display (children here))` ``` #lang pollen/markup ◊meta['template: "simple.html"] ◊meta['title: "Index"] ``` ``` <html> <head> </head> <body> ◊(display here) ◊(display (siblings here)) ◊(display (children here)) </body> </html> ``` Seems like the template is seeing an autogenerated pagetree rather than the one I'm supplying.

I agree this is annoying, but it’s not exactly a bug. You’re asking for siblings and children of the current source file. But you haven’t defined an explicit pagetree, so Pollen relies on the auto-generated pagetree, which is a “listing of files in the directory”.

Your objection, then, is that the directory listing ends up with files in it that shouldn’t count as siblings or children, such as pollen.cache. I agree, but I also don’t know how to make it better. Pollen is meant to be open-ended. Thus, what files are relevant is a project-dependent issue, known only to you.

For instance, suppose we said that Pollen should ignore all files beginning with a dot. But what if someone wants a project made of dotfiles? Also, cross-platform is an issue. The files that should be typically ignored on OS X are not the ones that should be ignored on Windows, and so forth. So relying on a simple directory-list offers the fewest surprises (even if overinclusive).

(PS I could probably make this reasoning clearer in the pollen/pagetree docs.)

If you have a better suggestion, I’m open to it.

I concede, however, that there’s never any reason to include the pollen.cache file. So I will update the auto-pagetree to omit that.

I agree this is annoying, but it’s not exactly a bug. You’re asking for `siblings` and `children` of the current source file. But you haven’t defined an explicit pagetree, so Pollen relies on the auto-generated pagetree, which is a [“listing of files in the directory”](http://pkg-build.racket-lang.org/doc/pollen/first-tutorial.html#%28part._.Using_the_dashboard%29). Your objection, then, is that the directory listing ends up with files in it that shouldn’t count as `siblings` or `children`, such as `pollen.cache`. I agree, but I also don’t know how to make it better. Pollen is meant to be open-ended. Thus, what files are relevant is a project-dependent issue, known only to you. For instance, suppose we said that Pollen should ignore all files beginning with a dot. But what if someone _wants_ a project made of dotfiles? Also, cross-platform is an issue. The files that should be typically ignored on OS X are not the ones that should be ignored on Windows, and so forth. So relying on a simple `directory-list` offers the fewest surprises (even if overinclusive). (PS I could probably make this reasoning clearer in the `pollen/pagetree` docs.) If you have a better suggestion, I’m open to it. I concede, however, that there’s never any reason to include the `pollen.cache` file. So I will update the auto-pagetree to omit that.

Actually directory-list already ignores files with a leading dot. Bad example, but the point remains: filtering the directory list for the automatic pagetree starts to creep into magic behavior.

Actually `directory-list` already ignores files with a leading dot. Bad example, but the point remains: filtering the directory list for the automatic pagetree starts to creep into magic behavior.

I agree this is annoying, but it’s not exactly a bug. You’re asking for siblings and children of the current source file. But you haven’t defined an explicit pagetree, so Pollen relies on the auto-generated pagetree, which is a “listing of files in the directory”.

My argument would be that I am defining an explicit pagetree: that's what's being passed to render-pagetree in the following:

(render-pagetree '(pagetree-root (index.html test.html)))

> I agree this is annoying, but it’s not exactly a bug. You’re asking for siblings and children of the current source file. But you haven’t defined an explicit pagetree, so Pollen relies on the auto-generated pagetree, which is a “listing of files in the directory”. My argument would be that I _am_ defining an explicit pagetree: that's what's being passed to `render-pagetree` in the following: ``` (render-pagetree '(pagetree-root (index.html test.html))) ```

I don't think it's expected that...

(render-pagetree "index.ptree")

where index.ptree is

#lang pollen

◊index.html{
a.html
b.html
}
c.html

...should differ from:

(render-pagetree '(pagetree-root (index.html a.html b.html) c.html))

I don't think it's expected that... ``` (render-pagetree "index.ptree") ``` where `index.ptree` is ``` #lang pollen ◊index.html{ a.html b.html } c.html ``` ...should differ from: ``` (render-pagetree '(pagetree-root (index.html a.html b.html) c.html)) ```

OK, I misunderstood. Let me give this another look.

The other thing was worth improving, and now it is improved, but I see your problem is different ;)

When you pass a pagetree to render-pagetree, it’s being used as a listing of files to render.

Then, when those files are actually rendered — which is happening in a separate evaluation environment — they look for a file-based pagetree, don’t find one, and thus fall back on the automatic pagetree.

So to make your example work, the pagetree passed to render-pagetree should flow all the way to the rendering environment.

I can see the logic, but I’ll need to consider the possible unintended consequences.

The other thing was worth improving, and now it is improved, but I see your problem is different ;) When you pass a pagetree to `render-pagetree`, it’s being used as a listing of files to render. Then, when those files are actually rendered — which is happening in a separate evaluation environment — they look for a file-based pagetree, don’t find one, and thus fall back on the automatic pagetree. So to make your example work, the pagetree passed to `render-pagetree` should flow all the way to the rendering environment. I can see the logic, but I’ll need to consider the possible unintended consequences.

The pagetree is meant as a general abstraction for describing relationships between source files, without mandating the end result. The pagetree can be used in a number of roles — for instance, within Pollen it’s used to define a navigation tree, or define a set of files to be rendered. (In the future, others might come up with other ways to use it or extend it — e.g., it could hold metadata about source files.)

I understand your use case, but here’s a counterexample (which I’ve already come across in my own work). Suppose I have an index.ptree that describes the navigation for a project. And a weekly.ptree that describes a subset of pages that I want to refresh weekly. So when I do (render-pagetree "weekly.ptree"), I definitely do not want weekly.ptree to be treated as the navigational pagetree. Two pagetrees; two roles. My intuition is that they should remain distinct.

But can you elaborate on why you want to be able to specify the navigation pagetree in an imperative manner (vs. just writing it to a file)? Perhaps there can be an affordance for doing this, though via a different route.

The pagetree is meant as a general abstraction for describing relationships between source files, without mandating the end result. The pagetree can be used in a number of roles — for instance, within Pollen it’s used to define a navigation tree, or define a set of files to be rendered. (In the future, others might come up with other ways to use it or extend it — e.g., it could hold metadata about source files.) I understand your use case, but here’s a counterexample (which I’ve already come across in my own work). Suppose I have an `index.ptree` that describes the navigation for a project. And a `weekly.ptree` that describes a subset of pages that I want to refresh weekly. So when I do `(render-pagetree "weekly.ptree")`, I definitely do **not** want `weekly.ptree` to be treated as the navigational pagetree. Two pagetrees; two roles. My intuition is that they should remain distinct. But can you elaborate on why you want to be able to specify the navigation pagetree in an imperative manner (vs. just writing it to a file)? Perhaps there can be an affordance for doing this, though via a different route.

So my use case, which I've mentioned in previous issues, is generating per category index pages for my blog. I store categories in the meta of each file. Then read all categories in from all files, remove-duplicates and generate an index page for each one.

I've just made the code for my site public, you can see it here (feel free to comment on any horrors you see in there!). I've gotten around the difference in behaviour by just "manually" writing the pagetree to a .ptree.

As a counterargument to your example, yes you don't want weekly.ptree used for navigation because it's not really a tree, it's a flat list of files. A tree by definition encodes the hierarchical relationship between elements of that tree.

If (pagetree-render "something.ptree") gives a different result from (render-page '(pagetree-root (...))) then at least the documentation has to reflect that. Maybe there should be an option to pagetree-render that you want this pagetree to be used for navigation, i.e. the pagetree that will be seen by children, siblings, next, etc.

So my use case, which I've mentioned in previous issues, is generating per category index pages for my blog. I store categories in the meta of each file. Then read all categories in from all files, `remove-duplicates` and generate an index page for each one. I've just made the code for my site public, you can see it [here](https://github.com/malcolmstill/mstill.io) (feel free to comment on any horrors you see in there!). I've gotten around the difference in behaviour by just "manually" writing the `pagetree` to a `.ptree`. As a counterargument to your example, yes you don't want `weekly.ptree` used for navigation because it's not really a _tree_, it's a flat list of files. A tree by definition encodes the hierarchical relationship between elements of that tree. If `(pagetree-render "something.ptree")` gives a different result from `(render-page '(pagetree-root (...)))` then at least the documentation has to reflect that. Maybe there should be an option to `pagetree-render` that you want this `pagetree` to be used for navigation, i.e. the `pagetree` that will be seen by `children`, `siblings`, `next`, etc.

Off topic but you might be interested in a wee library I've put together for numbering sections, figures, etc. and referencing these numbers. It's here.

Off topic but you might be interested in a wee library I've put together for numbering sections, figures, etc. and referencing these numbers. It's [here](https://github.com/malcolmstill/pollen-count).

As a counterargument to your example, yes you don't want weekly.ptree used for navigation because it's not really a tree, it's a flat list of files. A tree by definition encodes the hierarchical relationship between elements of that tree.

True, a list isn’t a tree, but #lang pollen/ptree treats flat lists as valid pagetrees (and you can pass nested structures to render-pagetree and it will just flatten them)

If (pagetree-render "something.ptree") gives a different result from (render-page '(pagetree-root (...))) then at least the documentation has to reflect that.

Yes.

> As a counterargument to your example, yes you don't want weekly.ptree used for navigation because it's not really a tree, it's a flat list of files. A tree by definition encodes the hierarchical relationship between elements of that tree. True, a list isn’t a tree, but `#lang pollen/ptree` treats flat lists as valid pagetrees (and you can pass nested structures to `render-pagetree` and it will just flatten them) > If (pagetree-render "something.ptree") gives a different result from (render-page '(pagetree-root (...))) then at least the documentation has to reflect that. Yes.

Off topic but you might be interested in a wee library I've put together for numbering sections, figures, etc. and referencing these numbers. It's here.

Great! I will look later.

> Off topic but you might be interested in a wee library I've put together for numbering sections, figures, etc. and referencing these numbers. It's here. Great! I will look later.

Off topic but you might be interested in a wee library I've put together for numbering sections, figures, etc. and referencing these numbers. It's here.
Great! I will look later.

I've written a blog post that might help explain usage until I get round to figuring out how to write Racket documentation.

> > Off topic but you might be interested in a wee library I've put together for numbering sections, figures, etc. and referencing these numbers. It's here. > > Great! I will look later. I've written [a blog post](http://mstill.io/blog/pollen-count.html) that might help explain usage until I get round to figuring out how to write Racket documentation.

pollen-count — cute

`pollen-count` — cute

I thought it was clever :)

pollen-count is cool, thanks for that. I think hierarchical numbering is likely to be a recurring request. I’m going to study your approach further. I may have some suggestions for generalizing it.

`pollen-count` is cool, thanks for that. I think hierarchical numbering is likely to be a recurring request. I’m going to study your approach further. I may have some suggestions for generalizing it.

Labels Milestones

Unexpected render-pagetree behaviour #78