Handling of tag functions within Pollen Markdown/Authoring mode #9

The "issue" or concern is how default tag function behavior is/is not handled in markdown authoring mode, and it's implications for using pollen.

When in markdown mode (*.poly.pmd),

◊(sidebar "hi!")

renders to:

(p rsquo "(sidebar " ldquo "hi!" rdquo ")")

In pollen markup mode, (*.poly.pm),

◊(sidebar "hi!")

renders to :

(sidebar "hi!")

I can imagine that this is related to the rule that tag functions in markdown authoring mode must render to markdown, however I am curious why the default tag function behavior does not apply. (or what the loss would be in changing it to apply)

In terms of the output X-expressions, if the default tag function behavior applied, then tag functions could be used to make a user extensible version of markdown. (which then using other parts of pollen could be used to have a relatively more readable source render to X-expression then convert to HTML+CSS and LaTeX)

To further the above example, consider the following .pm:

#lang pollen

◊(h1 "Barticle Title")
◊(p "The wonderful "◊(em "second") " part of the article.")
◊(sidebar "hi!")

which renders to:

(root
  (h1 "Barticle Title")
  (p "The wonderful " (em "second") " part of the article.")
  (sidebar "hi!"))

compared to the following markdown:

#lang pollen

Barticle Title
==============
 
The wonderful _second_ part of the article.

◊(sidebar "hi!")

which renders to:

(root
  (h1 ((id "barticle-title")) "Barticle Title")
  (p "The wonderful " (em "second") " part of the article.")
  (p rsquo "(sidebar " ldquo "hi!" rdquo ")"))

however, if the default tag function were obeyed even in markdown mode, it might be:

(root
  (h1 ((id "barticle-title")) "Barticle Title")
  (p "The wonderful " (em "second") " part of the article.")
  (sidebar "hi!"))

which, at least given my neophyte understanding of X-expressions, would be easier to work with. It would also seem to fulfill the goal of markdown authoring mode, which I presume to be to serve as a gateway drug to fuller use of pollen markup and semantic tags.

As possibly relevant context, I have approval from my supervisor to write a project in "70% markdown with some customization to allow for website and PDF to both look great", but not to use a new markup language. So I would happily ditch markdown entirely, however the difference in readability for everyone else in my (modestly tech savvy government) office threatens to rule out pollen before I get much further.

The "issue" or concern is how default tag function behavior is/is not handled in markdown authoring mode, and it's implications for using pollen. When in markdown mode (*.poly.pmd), ```◊(sidebar "hi!")``` renders to: ```(p rsquo "(sidebar " ldquo "hi!" rdquo ")")``` In pollen markup mode, (*.poly.pm), ```◊(sidebar "hi!")``` renders to : ```(sidebar "hi!")``` I can imagine that this is related to the rule that tag functions in markdown authoring mode must render to markdown, however I am curious why the default tag function behavior does not apply. (or what the loss would be in changing it to apply) In terms of the output X-expressions, if the default tag function behavior applied, then tag functions could be used to make a user extensible version of markdown. (which then using other parts of pollen could be used to have a relatively more readable source render to X-expression then convert to HTML+CSS and LaTeX) To further the above example, consider the following .pm: ``` #lang pollen ◊(h1 "Barticle Title") ◊(p "The wonderful "◊(em "second") " part of the article.") ◊(sidebar "hi!") ``` which renders to: ``` (root (h1 "Barticle Title") (p "The wonderful " (em "second") " part of the article.") (sidebar "hi!")) ``` compared to the following markdown: ``` #lang pollen Barticle Title ============== The wonderful _second_ part of the article. ◊(sidebar "hi!") ``` which renders to: ``` (root (h1 ((id "barticle-title")) "Barticle Title") (p "The wonderful " (em "second") " part of the article.") (p rsquo "(sidebar " ldquo "hi!" rdquo ")")) ``` however, if the default tag function were obeyed even in markdown mode, it might be: ``` (root (h1 ((id "barticle-title")) "Barticle Title") (p "The wonderful " (em "second") " part of the article.") (sidebar "hi!")) ``` which, at least given my neophyte understanding of X-expressions, would be easier to work with. It would also seem to fulfill the goal of markdown authoring mode, which I presume to be to serve as a gateway drug to fuller use of pollen markup and semantic tags. As possibly relevant context, I have approval from my supervisor to write a project in "70% markdown with some customization to allow for website and PDF to both look great", but not to use a new markup language. So I would happily ditch markdown entirely, however the difference in readability for everyone else in my (modestly tech savvy government) office threatens to rule out pollen before I get much further.

I am curious why the default tag function behavior does not apply. (or what the loss would be in changing it to apply)

The source of a pollen/markdown file is eventually passed through an actual Markdown parser. Your question seems to boil down to “how can I insert expressions that are exempt from this parsing?” That would require changes to the Markdown parser to be able to parse “things escaped from Markdown” which is not itself a concept inherent in Markdown.

So I would recommend seeing if you can lean into the Markdown parser rather than away from it. Meaning, you could adopt a little Markdown notation convention for your sidebar:

#lang pollen/markdown

Barticle Title
==============
 
The wonderful _second_ part of the article.


Sidebar: Hello world!
=====================


```
Sidebar: Hello world!
```

Which would become:

'(root
  (h1 ((id "barticle-title")) "Barticle Title")
  (p "The wonderful " (em "second") " part of the article.")
  (h1 ((id "sidebar-hello-world")) "Sidebar: Hello world!")
  (pre (code "Sidebar: Hello world!")))

And now you can do some further processing (e.g., traverse this X-expression, converting things that are marked as sidebars to actual (sidebar ···) expressions)

As a side benefit, your project would now be 100% Markdown syntax, and you’ve shifted your customizations into the semantic layer.

> I am curious why the default tag function behavior does not apply. (or what the loss would be in changing it to apply) The source of a `pollen/markdown` file is eventually passed through an actual Markdown parser. Your question seems to boil down to “how can I insert expressions that are exempt from this parsing?” That would require changes to the Markdown parser to be able to parse “things escaped from Markdown” which is not itself a concept inherent in Markdown. So I would recommend seeing if you can lean into the Markdown parser rather than away from it. Meaning, you could adopt a little Markdown notation convention for your sidebar: ```` #lang pollen/markdown Barticle Title ============== The wonderful _second_ part of the article. Sidebar: Hello world! ===================== ``` Sidebar: Hello world! ``` ```` Which would become: ``` '(root (h1 ((id "barticle-title")) "Barticle Title") (p "The wonderful " (em "second") " part of the article.") (h1 ((id "sidebar-hello-world")) "Sidebar: Hello world!") (pre (code "Sidebar: Hello world!"))) ``` And now you can do some further processing (e.g., traverse this X-expression, converting things that are marked as sidebars to actual `(sidebar ···)` expressions) As a side benefit, your project would now be 100% Markdown syntax, and you’ve shifted your customizations into the semantic layer.

I think the current behavior makes the most sense.

If you really want to escape to Pollen in Markdown authoring mode, you can still do so, you just have to define all such tag functions explicitly.

Another trick to take note of…not sure if this helps or not, but HTML is valid within Markdown, and any HTML produced in your Markdown file will get made into "clean" x-expressions:

#lang pollen/markdown

◊(define (sidebar str) (format "<dude>~a</dude>" str))

Barticle Title
==============
 
The wonderful _second_ part of the article.

◊sidebar{hi!} or ◊(sidebar "Hi!")

Here's some <arbitrary class="strange">more</arbitrary>.

becomes:

'(root
  (h1 ((id "barticle-title")) "Barticle Title")
  (p "The wonderful " (em "second") " part of the article.")
  (p (dude "hi!") " or " (dude "Hi!"))
  (p "Here" rsquo "s some " (arbitrary ((class "strange")) "more") "."))

So in Markdown authoring mode it might make sense to think of HTML itself as providing the "default tag" behavior you seek...?

I think the current behavior makes the most sense. If you really want to escape to Pollen in Markdown authoring mode, you can still do so, you just have to define all such tag functions explicitly. Another trick to take note of…not sure if this helps or not, but HTML is valid within Markdown, _and_ any HTML produced in your Markdown file will get made into "clean" x-expressions: ``` #lang pollen/markdown ◊(define (sidebar str) (format "<dude>~a</dude>" str)) Barticle Title ============== The wonderful _second_ part of the article. ◊sidebar{hi!} or ◊(sidebar "Hi!") Here's some <arbitrary class="strange">more</arbitrary>. ``` becomes: ``` '(root (h1 ((id "barticle-title")) "Barticle Title") (p "The wonderful " (em "second") " part of the article.") (p (dude "hi!") " or " (dude "Hi!")) (p "Here" rsquo "s some " (arbitrary ((class "strange")) "more") ".")) ``` So in Markdown authoring mode it might make sense to think of HTML itself as providing the "default tag" behavior you seek...?

Thanks! Both of those helped considerably. Joel's method works quite well in this context, and nests nicely. (e.g. a section can be in a pollen defined tag but otherwise be conventional markdown). I think I can just doctor up a function to define the list of functions I'll be using.

Matthew's method might work well for a few functions that need to be more "markdown" looking, such as an irritating Q&A section format.

Thanks! Both of those helped considerably. Joel's method works quite well in this context, and nests nicely. (e.g. a section can be in a pollen defined tag but otherwise be conventional markdown). I think I can just doctor up a function to define the list of functions I'll be using. Matthew's method might work well for a few functions that need to be more "markdown" looking, such as an irritating Q&A section format.

Labels Milestones

Handling of tag functions within Pollen Markdown/Authoring mode #9