How to use Pagetree Navigation with Subdirectories #7

A project I'm working on would be much more manageable if I could use subdirectories with the pagetree. When I started to implement this, the navigation I expect with (next here), for example, stopped working. It just returned a blank string.

I read from this thread that I will need to do something like this in my pollen.rkt file:

(define here-path (hash-ref metas 'here-path))
;; `here-path` is an absolute path, so if your pagetree nodes are relative to project root,
;; you'll also want to make this one relative the same way.
(require racket/path) ; for `find-relative-path`. 
(define here-path-relative (find-relative-path here-path (current-project-root)))
;; every node in a pagetree is a symbol
(define here-path-pagenode (string->symbol (path->string here-path-relative)))
;; having made our special pagenode, now we can query the pagetree
(define next-path-pagenode (next here-path-pagenode)) 
;; then we convert the result back to an output path
(define next-url (->output-path (build-path (current-project-root) (symbol->string next-path-pagenode))))

with the proviso that if this code lives inside pollen.rkt,:

you can pass metas as an argument, or use the current-metas parameter.

Could someone help me with how to do that? At the moment I get unbound identifier in metas inside DrRacket.

A project I'm working on would be much more manageable if I could use subdirectories with the pagetree. When I started to implement this, the navigation I expect with `(next here)`, for example, stopped working. It just returned a blank string. I read from [this thread](https://groups.google.com/forum/#!searchin/pollenpub/pagetree$20subdirectories|sort:date/pollenpub/UxNKyP5yrFo/1jbp_7ahAAAJ) that I will need to do something like this in my pollen.rkt file: ``` (define here-path (hash-ref metas 'here-path)) ;; `here-path` is an absolute path, so if your pagetree nodes are relative to project root, ;; you'll also want to make this one relative the same way. (require racket/path) ; for `find-relative-path`. (define here-path-relative (find-relative-path here-path (current-project-root))) ;; every node in a pagetree is a symbol (define here-path-pagenode (string->symbol (path->string here-path-relative))) ;; having made our special pagenode, now we can query the pagetree (define next-path-pagenode (next here-path-pagenode)) ;; then we convert the result back to an output path (define next-url (->output-path (build-path (current-project-root) (symbol->string next-path-pagenode)))) ``` with the proviso that if this code lives inside pollen.rkt,: > you can pass `metas` as an argument, or use the `current-metas` parameter. Could someone help me with how to do that? At the moment I get *unbound identifier in metas* inside DrRacket.

I literally ran into the same problem today. The code at the linked conversation runs in the pollen template file, not the pollen.rkt file, otherwise you can't get to the metas.

If you add the lozenges, you can output the defined variables in the template.

There's a little more info here: https://docs.racket-lang.org/pollen/pollen-command-syntax.html#%28part._.Inserting_metas%29 . On the page it shows a way to include it in a racket source file, but I couldn't get that to work. I tried using the current-metas, but when I do it doesn't have the 'here-path key. So I'm more or less in the same boat. :)

I literally ran into the same problem today. The code at the linked conversation runs in the pollen template file, not the pollen.rkt file, otherwise you can't get to the metas. If you add the lozenges, you can output the defined variables in the template. There's a little more info here: https://docs.racket-lang.org/pollen/pollen-command-syntax.html#%28part._.Inserting_metas%29 . On the page it shows a way to include it in a racket source file, but I couldn't get that to work. I tried using the current-metas, but when I do it doesn't have the 'here-path key. So I'm more or less in the same boat. :)

Thank you, @McSackeri

I think it would be possible to have this inside pollen.rkt, but that metas would need to be passed from the template as an argument. I will need to have pages in sub-directories for multiple page templates, so it would be more DRY to have the code once inside the .rkt file rather than in every template file that needs it. However for the timebeing that's not essential, as I just would like to get this working in the next two weeks for a presentation I'm making using this content. (Part of the job of this project is building slides for revealjs, by the way.)

Therefore I'm ploughing on with having the code inside template.html.p just to try to get it working, even though my eventual goal is to have it working inside pollen.rkt. I have got something to work, but it's still not functional.

◊(define here-path (hash-ref metas 'here-path))
◊(local-require racket/path) 
◊(define here-path-relative (find-relative-path **(current-project-root) here-path)**)
◊(define here-path-pagenode (string->symbol (path->string here-path-relative)))
◊(define next-path-pagenode (next here-path-pagenode))

The results from the rendered html file:

here-path V:\myAppFolder\content\temp1.html.pm.
current-project-root V:\myAppFolder\content\
here-path-relative temp1.html.pm
here-path-pagenode temp1.html.pm
next-path-pagenode

So it seems to be working down as far as next-path-pagenode, which just returns nothing at all.

For the record, (next here) returns temp2.html, so there is actually a next page in the pagetree. Another note is that these two temp.html.pm files are in the project root, but the same results are returned from pages in subfolders, except that the ones in subfolders return nothing for (next here), so trying to get this working with files in the root folder first seemed like the logical thing to do.

Thank you, @McSackeri I think it would be possible to have this inside pollen.rkt, but that metas would need to be passed from the template as an argument. I will need to have pages in sub-directories for multiple page templates, so it would be more DRY to have the code once inside the .rkt file rather than in every template file that needs it. However for the timebeing that's not essential, as I just would like to get this working in the next two weeks for a presentation I'm making using this content. (Part of the job of this project is building slides for [revealjs](http://revealjs.com/), by the way.) Therefore I'm ploughing on with having the code inside template.html.p just to try to get it working, even though my eventual goal is to have it working inside pollen.rkt. I have got something to work, but it's still not functional. ``` ◊(define here-path (hash-ref metas 'here-path)) ◊(local-require racket/path) ◊(define here-path-relative (find-relative-path **(current-project-root) here-path)**) ◊(define here-path-pagenode (string->symbol (path->string here-path-relative))) ◊(define next-path-pagenode (next here-path-pagenode)) ``` The results from the rendered html file: - here-path V:\myAppFolder\content\temp1.html.pm. - current-project-root V:\myAppFolder\content\ - here-path-relative temp1.html.pm - here-path-pagenode temp1.html.pm - next-path-pagenode So it seems to be working down as far as next-path-pagenode, which just returns nothing at all. For the record, `(next here)` returns temp2.html, so there is actually a next page in the pagetree. Another note is that these two temp.html.pm files are in the project root, but the same results are returned from pages in subfolders, except that the ones in subfolders return nothing for `(next here)`, so trying to get this working with files in the root folder first seemed like the logical thing to do.

Ok, so after down and back up the rabbit hole, I think I found an ugly, but workable hack. I'm actually not sure if we're trying to do the same thing, but our symptoms are very similar. It seems that the navigation for the pagetree really wants to work from one location on the site, not from multiple subfolders. So when I put subfolder names in the root index.ptree, here, (previous here), and (next here) return values the way you expect them. But the problem is if you're in a page rendered from a subfolder, they keep the subfolder name, so you can't actually use them to navigate because they try to take you to yet another subfolder of the same name (ie if you're in "folder/filename.html", (next here) will return "folder/filename2.html" and your link will look like "folder/folder/filename2.html").

But if you don't put them in your root directory pagetree, and instead put them in a separate pagetree in the subfolder, the navigation tags won't work. The function "here" will return "folder/filename.html", but (previous here) and (next here) return nothing.

Anyway after trying a number of things, this actually works pretty well in the template file:

◊(local-require racket/path)
◊(define heer (file-name-from-path (symbol->string here)))

Then you can replace "here" with "heer" like so:

<a href="◊|(previous heer)|">◊|(previous heer)|</a>  <br/>
<a href="◊|(next heer)|">◊|(next heer)|</a> <br/>

Literally all I'm doing is extracting the file name without the folder from "here" and using it as a pagenode for the "previous", and "next" functions which gets them to see the correct nodes in the subfolder pagetree. There's probably a better way to do this, but I'm still a Racket/Pollen novice, and it's the best my brain can handle at this point. :)

Ok, so after down and back up the rabbit hole, I think I found an ugly, but workable hack. I'm actually not sure if we're trying to do the same thing, but our symptoms are very similar. It seems that the navigation for the pagetree really wants to work from one location on the site, not from multiple subfolders. So when I put subfolder names in the root index.ptree, here, **(previous here)**, and (**next here)** return values the way you expect them. But the problem is if you're in a page rendered from a subfolder, they keep the subfolder name, so you can't actually use them to navigate because they try to take you to yet another subfolder of the same name _(ie if you're in "folder/filename.html", **(next here)** will return "folder/filename2.html" and your link will look like "folder/folder/filename2.html")_. But if you don't put them in your root directory pagetree, and instead put them in a separate pagetree in the subfolder, the navigation tags won't work. The function "here" will return "folder/filename.html", but (previous here) and (next here) return nothing. Anyway after trying a number of things, this actually works pretty well in the template file: ``` ◊(local-require racket/path) ◊(define heer (file-name-from-path (symbol->string here))) ``` Then you can replace "here" with "heer" like so: ```html <a href="◊|(previous heer)|">◊|(previous heer)|</a> <br/> <a href="◊|(next heer)|">◊|(next heer)|</a> <br/> ``` Literally all I'm doing is extracting the file name without the folder from "here" and using it as a pagenode for the "previous", and "next" functions which gets them to see the correct nodes in the subfolder pagetree. There's probably a better way to do this, but I'm still a Racket/Pollen novice, and it's the best my brain can handle at this point. :)

I think you’re on the right track. A lot of this is just creating a simple, consistent convention for handling paths in your project so you’re not switching between relative and absolute paths at the wrong moments.

For instance, on Beautiful Racket I have global prev/next navigation that traverses subdirectories correctly. The way I do this is by having one index.ptree at the top level. The pagenodes are represented as relative to the top level. A partial listing of the output:

'(pagetree-root
  (index.html
   foreword.html
   introduction.html
   setup.html
   acknowledgments.html
   about-the-author.html
   legal.html
   how-to-pay.html
   why-you-should-pay.html
   (tutorials-group
    (stacker/intro.html
     stacker/why-make-languages.html
     stacker/setup.html
     stacker/the-reader.html
     stacker/the-expander.html
     stacker/recap.html
     stacker/source-listing.html)
    (funstacker/intro.html
     funstacker/project-setup.html
     funstacker/the-rewrite.html
     funstacker/recap.html
     funstacker/source-listing.html)
    (stackerizer/intro.html
     stackerizer/specification-and-setup.html
     stackerizer/the-expander.html
     stackerizer/recap.html
     stackerizer/source-listing.html)
···

Then in my template.html.p — there is only one, also at the top level — I make sure that my prev/next links are always rendered as absolute URLs. Meaning, they start with "/". Then each pagenode provides the rest of the path. Because I’m using absolute URLs, they work the same whether they appear on a top-level page or subdirectory page.

“But dude — that’s not DRY at all! You’re retyping all those subdirectory names!” Ah, but I’m not. Recall that a pagetree source is just another Pollen source, and you can have functions and macros in it as usual. So the index.ptree source looks like this:

#lang pollen

◊index.html{
 foreword.html
 introduction.html
 setup.html
 acknowledgments.html
 about-the-author.html
 legal.html
 how-to-pay.html
 why-you-should-pay.html
 ◊tutorials-group{
  ◊(splice-subdir stacker)
  ◊(splice-subdir funstacker)
  ◊(splice-subdir stackerizer)
  ◊(splice-subdir bf)
  ◊(splice-subdir jsonic)
  ◊(splice-subdir jsonic-2)
  ◊(splice-subdir jsonic-3)
  ◊(splice-subdir wires)
  ◊(splice-subdir basic)
  ◊(splice-subdir basic-2)
  ◊(splice-subdir basic-3)
  ◊(splice-subdir-dev hdl)
 }
}
···

Where splice-subdir is a little macro that goes through and adds all the files from the subdirectory to the pagetree, and also prepends the subdirectory to the path. So DRY it hurts.

I think you’re on the right track. A lot of this is just creating a simple, consistent convention for handling paths in your project so you’re not switching between relative and absolute paths at the wrong moments. For instance, on [Beautiful Racket](//beautifulracket.com) I have global prev/next navigation that traverses subdirectories correctly. The way I do this is by having one `index.ptree` at the top level. The pagenodes are represented as relative to the top level. A partial listing of the output: ``` '(pagetree-root (index.html foreword.html introduction.html setup.html acknowledgments.html about-the-author.html legal.html how-to-pay.html why-you-should-pay.html (tutorials-group (stacker/intro.html stacker/why-make-languages.html stacker/setup.html stacker/the-reader.html stacker/the-expander.html stacker/recap.html stacker/source-listing.html) (funstacker/intro.html funstacker/project-setup.html funstacker/the-rewrite.html funstacker/recap.html funstacker/source-listing.html) (stackerizer/intro.html stackerizer/specification-and-setup.html stackerizer/the-expander.html stackerizer/recap.html stackerizer/source-listing.html) ··· ``` Then in my `template.html.p` — there is only one, also at the top level — I make sure that my prev/next links are always rendered as absolute URLs. Meaning, they start with "/". Then each pagenode provides the rest of the path. Because I’m using absolute URLs, they work the same whether they appear on a top-level page or subdirectory page. “But dude — that’s not DRY at all! You’re retyping all those subdirectory names!” Ah, but I’m not. Recall that a pagetree source is just another Pollen source, and you can have functions and macros in it as usual. So the `index.ptree` source looks like this: ``` #lang pollen ◊index.html{ foreword.html introduction.html setup.html acknowledgments.html about-the-author.html legal.html how-to-pay.html why-you-should-pay.html ◊tutorials-group{ ◊(splice-subdir stacker) ◊(splice-subdir funstacker) ◊(splice-subdir stackerizer) ◊(splice-subdir bf) ◊(splice-subdir jsonic) ◊(splice-subdir jsonic-2) ◊(splice-subdir jsonic-3) ◊(splice-subdir wires) ◊(splice-subdir basic) ◊(splice-subdir basic-2) ◊(splice-subdir basic-3) ◊(splice-subdir-dev hdl) } } ··· ``` Where `splice-subdir` is a little macro that goes through and adds all the files from the subdirectory to the pagetree, and also prepends the subdirectory to the path. So DRY it hurts.

PS. The splice-subdir macro actually checks for the presence of another index.ptree in each subdirectory and hoists it into the top-level index.ptree by looking at its doc export. That way, I get the benefit of index.ptree in each subdirectory for ordering the pages, but all the navigation is computed from the top-level index.ptree.

PS. The `splice-subdir` macro actually checks for the presence of another `index.ptree` in each subdirectory and hoists it into the top-level `index.ptree` by looking at its `doc` export. That way, I get the benefit of `index.ptree` in each subdirectory for ordering the pages, but all the navigation is computed from the top-level `index.ptree`.

I am really grateful for the help on this. I am genuinely inspired and motivated to learn Racket properly so that I don't any longer feel feel that I'm just poking around with stuff I don't really understand.

This morning I methodically tried to work this out and found something related to using Pollen on Windows. What I discovered was, as far as I can tell, that pollen converts a forward slash to a backslash for here, but for previous here and other expressions, the forward slashes are not converted, the pagenode isn't found, and the expression returns nothing.

Here is how I discovered this, with a minimal working example. The ptree is this:

#lang pollen

1.html
◊2.html{
	sub/3.html
	sub/4.html
}

The relevant part of the template is this:

◊(define parent-page (parent here))
◊(define previous-page (previous here))
◊(define next-page (next here))

<div class="pollen-info">
<ul>
<li>The current page is called ◊|here|.</li>
<li>The previous is ◊|(previous here)|.</li>
<li>The next is ◊|(next here)|.</li>
</ul>
</div>

Everything works perfectly for 1.html and 2.html. In 2.html, the next page list item appears like so: sub/3.html. Note the forward slash.

In 3.html, the current page list item is rendered sub\3.html. Note the backslash.

When I define a new variable real-here and use that in the template instead, previous and next work as expected.

(define real-here (regexp-replace* #rx"[\\]" (symbol->string here) "/"))

It was only later in the day that it struck me, I'm so used to using forward slashes to indicate directories on anything server-related that I remembered, I'm using a Windows server. Changed the forward slashes to backslashes in the pagetree, and it all works fine right out of the box, without the need for a new real-here definition.

Finding this bug in my program has been a very fruitful learning journey for me, but it's cost a lot of hours and I wonder if there is a way to help other Windows users from running into this problem. Perhaps a small addendum at the 'flat.ptree' example (with facts/brennan) to use backslashes on Windows.

I am really grateful for the help on this. I am genuinely inspired and motivated to learn Racket properly so that I don't any longer feel feel that I'm just poking around with stuff I don't really understand. This morning I methodically tried to work this out and found something related to using Pollen on Windows. What I discovered was, as far as I can tell, that pollen converts a forward slash to a backslash for `here`, but for `previous here` and other expressions, the forward slashes are not converted, the pagenode isn't found, and the expression returns nothing. Here is how I discovered this, with a minimal working example. The ptree is this: ``` #lang pollen 1.html ◊2.html{ sub/3.html sub/4.html } ``` The relevant part of the template is this: ``` ◊(define parent-page (parent here)) ◊(define previous-page (previous here)) ◊(define next-page (next here)) <div class="pollen-info"> <ul> <li>The current page is called ◊|here|.</li> <li>The previous is ◊|(previous here)|.</li> <li>The next is ◊|(next here)|.</li> </ul> </div> ``` Everything works perfectly for 1.html and 2.html. In 2.html, the next page list item appears like so: `sub/3.html.` Note the forward slash. In 3.html, the current page list item is rendered `sub\3.html`. Note the backslash. When I define a new variable `real-here` and use that in the template instead, previous and next work as expected. ``` (define real-here (regexp-replace* #rx"[\\]" (symbol->string here) "/")) ``` It was only later in the day that it struck me, I'm so used to using forward slashes to indicate directories on anything server-related that I remembered, *I'm using a Windows server.* Changed the forward slashes to backslashes in the pagetree, and it all works fine right out of the box, without the need for a new `real-here` definition. Finding this bug in my program has been a very fruitful learning journey for me, but it's cost a lot of hours and I wonder if there is a way to help other Windows users from running into this problem. Perhaps a small addendum at [the 'flat.ptree' example (with facts/brennan)](https://docs.racket-lang.org/pollen/Pagetree.html#%28part._.Making_pagetrees_with_a_source_file%29) to use backslashes on Windows.

I concede I don’t work on Windows much — and I detest all filesystem programming, though files, we need ’em, right? — so I’m not averse to clarifications for Windows users. But yes, handling slash vs. backslash is part of the “simple, consistent convention for handling paths in your project” that you’d need to decide upfront.

As for the flat.ptree file, you should do it however you want. My intuition is that because the pagetree represents an “output” view of the world, it should probably reflect the path conventions on output. So if the project targets HTML, it makes sense to use HTML-style forward slashes. And handle the conversion from a platform path to an HTML path elsewhere. (This would have the added benefit of making the project portable between platforms, since nothing is Windows-centric.)

As for this:

(define real-here (regexp-replace* #rx"[\\]" (symbol->string here) "/"))

We’ve all done it, but processing path-like things with regex patterns can go very wrong. Racket has a set of path-manipulation functions that take care of the underlying ~~suffering~~ housekeeping.

I concede I don’t work on Windows much — and I detest all filesystem programming, though files, we need ’em, right? — so I’m not averse to clarifications for Windows users. But yes, handling slash vs. backslash is part of the “simple, consistent convention for handling paths in your project” that you’d need to decide upfront. As for the `flat.ptree` file, you should do it however you want. My intuition is that because the pagetree represents an “output” view of the world, it should probably reflect the path conventions on output. So if the project targets HTML, it makes sense to use HTML-style forward slashes. And handle the conversion from a platform path to an HTML path elsewhere. (This would have the added benefit of making the project portable between platforms, since nothing is Windows-centric.) As for this: ``` (define real-here (regexp-replace* #rx"[\\]" (symbol->string here) "/")) ``` We’ve all done it, but processing path-like things with regex patterns can go very wrong. Racket has a set of [path-manipulation functions](https://docs.racket-lang.org/reference/Manipulating_Paths.html) that take care of the underlying ~~suffering~~ housekeeping.

processing path-like things with regex patterns can go very wrong.

That's a very good point about not using the wrong tool for the job.

After a lot of fiddling with those path-manipulation functions, I decided to stick with backslashes, since that's what is working out of the box with next here and friends. Also, backslashes is what I see returned with parameters like current directory, and so sticking with that feels more like going with the flow.

> processing path-like things with regex patterns can go very wrong. That's a very good point about not using the wrong tool for the job. After a lot of fiddling with those path-manipulation functions, I decided to stick with backslashes, since that's what is working out of the box with `next here` and friends. Also, backslashes is what I see returned with parameters like [current directory](https://docs.racket-lang.org/reference/Filesystem.html#%28def._%28%28quote._~23~25kernel%29._current-directory%29%29), and so sticking with that feels more like going with the flow.

Labels Milestones

How to use Pagetree Navigation with Subdirectories #7