backlinks? #47

Hello all,

I am trying to figure out how, if possible, to create backlinks in Pollen.

If page A links to page B, then a 'back link' would be a link which goes from page B back to page A.

Several wiki engines support this feature and I would be very interested to know if anyone has implemented such a system for references, etc. I'm trying to avoid being sucked in by Roam Research. Any thoughts or ideas appreciated.

Hello all, I am trying to figure out how, if possible, to create [backlinks] in Pollen. > If page A links to page B, then a 'back link' would be a link which goes from page B back to page A. Several [wiki engines] support this feature and I would be very interested to know if anyone has implemented such a system for references, etc. I'm trying to avoid being sucked in by [Roam Research]. Any thoughts or ideas appreciated. [backlinks]: https://wiki.c2.com/?BackLink [wiki engines]: https://ikiwiki.info/backlinks/ [Roam Research]: https://roamresearch.com/

I tried to do this in the past. It was very painful, but perhaps I didn't approach the problem correctly. What you really want is a way to have several traversal passes just like Scribble.

You could approximate a multi-pass render with two ingredients:

a link index that gathers the links from all the pages and emits them in some convenient format (XML, JSON, S-expression, etc.) When you regenerate the whole project, you would generate this link index first. After that, when you update individual page files, you could incrementally update the link index.
if you wanted the backlinks, say, in the bottom of each page, you would add a function to your template.html.p that reads the links for a certain page from the link index and inserts them.

You could approximate a multi-pass render with two ingredients: 1) a link index that gathers the links from all the pages and emits them in some convenient format (XML, JSON, S-expression, etc.) When you regenerate the whole project, you would generate this link index first. After that, when you update individual page files, you could incrementally update the link index. 2) if you wanted the backlinks, say, in the bottom of each page, you would add a function to your `template.html.p` that reads the links for a certain page from the link index and inserts them.

Further to this multi-pass approach, consider how a Pollen document gets rendered:

Pollen “runs” your document as code which produces its doc and metas values.
The doc and metas get handed to a template, which has its own code for inserting those values into the rendered file.

One way you can get things to run somewhat faster is if each document does its own updates to the “link index” (I like to use a SQLite database file) during step 1. Then use code that runs during step 2 to collect information back out of the link index and insert it at the point the template is rendered.

You also have to think about dependencies a lot harder now. You edit some files and add a bunch of links into them. How do you ensure that all the linked-to pages get re-rendered properly with all the correct backlink information? You could just remember which links you added and re-render those pages manually. Or you could just rebuild the entire site every time you add links. Or you could use a makefile to describe a dependency tree and then, as part of your ‘link’ function (i.e., during step 1) do a touch of the file that is the target of the link (example), updating its timestamp and causing make to re-render the page.

Or, you could write a web server in Racket that provides an API for retrieving info from your link index, and update the backlinks dynamically with JavaScript on the client end. Boom, no more having to think about dependencies and build steps, all the cross-referency bits gets handled dynamically just like in a database-backed CMS. But all that complexity is conserved: now you have to think up front about how to provide and consume this API.

Further to this multi-pass approach, consider how a Pollen document gets rendered: 1. Pollen “runs” your document as code which produces its `doc` and `metas` values. 2. The `doc` and `metas` get handed to a template, which has its own code for inserting those values into the rendered file. One way you can get things to run somewhat faster is if each document does its own updates to the “link index” (I like to use a SQLite database file) during step 1. Then use code that runs during step 2 to collect information back _out_ of the link index and insert it at the point the template is rendered. You also have to think about dependencies a lot harder now. You edit some files and add a bunch of links into them. How do you ensure that all the linked-to pages get re-rendered properly with all the correct backlink information? You could just remember which links you added and re-render those pages manually. Or you could just rebuild the entire site every time you add links. Or you could use a [makefile](https://www.gnu.org/software/make/manual/make.html#Introduction) to describe a dependency tree and then, as part of your ‘link’ function (i.e., during step 1) do a `touch` of the file that is the target of the link ([example](https://thelocalyarn.com/code/artifact?udc=1&ln=126-135&name=cb0db902562bb7ca)), updating its timestamp and causing `make` to re-render the page. Or, you could write a web server in Racket that provides an API for retrieving info from your link index, and update the backlinks dynamically with JavaScript on the client end. Boom, no more having to think about dependencies and build steps, all the cross-referency bits gets handled dynamically just like in a database-backed CMS. But all that complexity is conserved: now you have to think up front about how to provide and consume this API.

@tlcu Please let me know if you solve this. I'm looking at Roam myself, as I need some sort of "graph-of-knowledge" notetaking tool, too. It would be cool to have it all in Pollen. How do you plan to handle the links themselves? Will you do any sort of UID, similar to how its done in the typical Zettelkasten method?

@tlcu Please let me know if you solve this. I'm looking at Roam myself, as I need some sort of "graph-of-knowledge" notetaking tool, too. It would be cool to have it all in Pollen. How do you plan to handle the links themselves? Will you do any sort of UID, similar to how its done in the typical Zettelkasten method?

@Eugleo I will certainly keep you informed. I plan on clearing out my work for this week and seeing what I can do over the weekend. At this point I am weighing adapting an existing solution (like Ikiwiki) against a from-scratch implementation. I of course agree that the knowledge base in Pollen and Racket would be very cool! I've created a repository here, where everything and everything is up for discussion (including the name :) ): https://github.com/tlcu/rome-research

I imagine that every link will be associated with a unique identifier, but I am happy to hear more so feel free to create an issue in the new repo. I have heard of Zettelkasten but am not overly familiar with its inner workings, feel free also to educate me.

I sincerely appreciate the remarks of everyone above, and will update this thread if I can find a neat solution.

> @tlcu Please let me know if you solve this. I'm looking at Roam myself, as I need some sort of "graph-of-knowledge" notetaking tool, too. It would be cool to have it all in Pollen. How do you plan to handle the links themselves? Will you do any sort of UID, similar to how its done in the typical Zettelkasten method? @Eugleo I will certainly keep you informed. I plan on clearing out my work for this week and seeing what I can do over the weekend. At this point I am weighing adapting an existing solution (like Ikiwiki) against a from-scratch implementation. I of course agree that the knowledge base in Pollen and Racket would be very cool! I've created a repository here, where everything and everything is up for discussion (including the name :) ): https://github.com/tlcu/rome-research I imagine that every link will be associated with a unique identifier, but I am happy to hear more so feel free to create an issue in the new repo. I have heard of Zettelkasten but am not overly familiar with its inner workings, feel free also to educate me. I sincerely appreciate the remarks of everyone above, and will update this thread if I can find a neat solution.

Labels Milestones

backlinks? #47