Feature request: server auto-refreshes page when save detected #35

Removing one step from the "save, switch apps, refresh" fandango would be helpful! Actually, this feature would reduce it to just "save, glance at browser", since you wouldn't need to switch apps at that point, if you have both visible.

I don’t oppose the idea. I just haven’t been able to figure out how to do it cleanly. If you have ideas, I’m open to them.

Since HTTP is stateless, I believe it would require embedding some logic in the page that would check the server intermittently for changed files. But since it’s a development / testing system, I’m reluctant to do that, because it would add code to the page that the developer didn’t put there (and potentially could interact in negative or mysterious ways with other code).

Beyond that, if you were editing a static HTML page and previewing it a browser, you’d have to engage in the same steps of “save, switch apps, refresh,” yes? Meaning, I’ve generally thought of this as an irreducible complexity of making web pages.

I don’t oppose the idea. I just haven’t been able to figure out how to do it cleanly. If you have ideas, I’m open to them. Since HTTP is stateless, I believe it would require embedding some logic in the page that would check the server intermittently for changed files. But since it’s a development / testing system, I’m reluctant to do that, because it would add code to the page that the developer didn’t put there (and potentially could interact in negative or mysterious ways with other code). Beyond that, if you were editing a static HTML page and previewing it a browser, you’d have to engage in the same steps of “save, switch apps, refresh,” yes? Meaning, I’ve generally thought of this as an irreducible complexity of making web pages.

Many web development systems I've worked with offer, in the debug/testing phases of development, just what you suggest: a little bit of JavaScript in the HTML that, either through AJAX or Websockets, can learn from the backend that it needs to refresh. Node.js devs have this tradition, which of course means emacs does too, via Skewer.

As for editing static HTML/CSS/JS, many text editors also offer live-preview intended to remove this "edit-save-refresh" cycle: Sublime, emacs 1 (built on top of Skewer) and 2, LightTable (which is a bit of a cheat: the entire editor is implemented in WebKit, and it can be made to show a tab that live-updates), and others I'm sure.

I'm not familiar with the specifics of how all these work, I've just used them without thinking. It's one of those things that can, in some little way, change how you work.

Many web development systems I've worked with offer, in the debug/testing phases of development, just what you suggest: a little bit of JavaScript in the HTML that, either through AJAX or Websockets, can learn from the backend that it needs to refresh. [Node.js](https://www.airpair.com/node.js/posts/top-10-mistakes-node-developers-make#1-2-automatic-browser-refresh) devs have this tradition, which of course means emacs does too, via [Skewer](http://nullprogram.com/blog/2012/10/31/). As for editing static HTML/CSS/JS, many text editors also offer live-preview intended to remove this "edit-save-refresh" cycle: [Sublime](https://github.com/dz0ny/LiveReload-sublimetext2), emacs [1](http://www.50ply.com/blog/2012/08/13/introducing-impatient-mode/) (built on top of Skewer) and [2](http://www.emacswiki.org/emacs/MozRepl), LightTable (which is a bit of a cheat: the entire editor is implemented in WebKit, and it can be made to show a tab that live-updates), and others I'm sure. I'm not familiar with the specifics of how all these work, I've just used them without thinking. It's one of those things that can, in some little way, change how you work.

Fair points, that I will consider further.

The other dimension is that unlike other web-publishing systems, where browser preview is the only option, the source files in Pollen are real code, and produce output that you can inspect in DrRacket (or the command line). I work in DrRacket, so I’m often doing exploratory editing & recompiling there to get a sense of things before looking at the output in a browser. Same thing when debugging. This involves a lot of incremental saving, so a live-preview system would be a constant state of reload. I guess it would have to be a mode that you toggle on or off.

Fair points, that I will consider further. The other dimension is that unlike other web-publishing systems, where browser preview is the only option, the source files in Pollen are real code, and produce output that you can inspect in DrRacket (or the command line). I work in DrRacket, so I’m often doing exploratory editing & recompiling there to get a sense of things before looking at the output in a browser. Same thing when debugging. This involves a lot of incremental saving, so a live-preview system would be a constant state of reload. I guess it would have to be a mode that you toggle on or off.

I’ve considered this idea occasionally, but I’m just not feeling enough enthusiasm to implement it. Sorry. I’m sure it would be nice, but it’s mostly magical UI sugar, and thus ranks low on the list of most valuable Pollen improvements. If someone wants to submit a pull request, great, I’ll reconsider it then.

@mbutterick I've found myself wanting this feature as well, and I'd love to take a crack at implementing it as a way to learn pollen internals a bit better. I see pollen/private/project-server.rkt, but do you recommend any other places to start tinkering? Or any background reading or resources on how the project server is implemented/ what it's based on?

@mbutterick I've found myself wanting this feature as well, and I'd love to take a crack at implementing it as a way to learn pollen internals a bit better. I see [`pollen/private/project-server.rkt`](https://github.com/mbutterick/pollen/blob/master/pollen/private/project-server.rkt), but do you recommend any other places to start tinkering? Or any background reading or resources on how the project server is implemented/ what it's based on?

As I did with #253, I’m willing to put the smallest sufficient amount of undocumented code into Pollen for you to experiment with this. Though I’ve always supposed most of the heavy lifting is on the client side. So this is more of a JS project than a Racket project. AFAICT the Pollen server could support this kind of reloading by adding a route that returns, say, the MD5 of a certain file on disk. The client would poll this route asynchronously and decide whether to request a full refresh.

It's been a while but I wrote a proof of concept of how this could work using server-sent events (MDN docs on SSE) that consisted of

a server component in Node.js, which should be readily portable to Racket (see server.js), and
a tiny Pollen+JavaScript bit that injects a bit of JS if being served with live-reload and that disappears otherwise: see the relevant lines of template.html.
- (There's some prose description of how this all works too.)

SSE is old-school these days. Next.js and Create-React-App and all these things use websockets for their hot-module reloading. I doubt there's much intrinsic advantage of SSE vs websockets for this little usecase, but the latter are much more useful and widespread so it's conceivable that Racket has nice websocket support that'll make the server component a breeze.

Though unfamiliar with the Racket ecosystem, I'm happy to advise.

It's been a while but I wrote a proof of concept of how this could work using server-sent events ([MDN docs on SSE](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events)) that consisted of - a server component in Node.js, which should be readily portable to Racket (see [server.js](https://github.com/fasiha/pollen-guide/blob/master/take3/server.js)), and - a tiny Pollen+JavaScript bit that injects a bit of JS if being served with live-reload and that disappears otherwise: see the relevant lines of [template.html](https://github.com/fasiha/pollen-guide/blob/master/take3/template.html#L7-L21). - (There's some [prose description](https://github.com/fasiha/pollen-guide#take-three-where-we-get-all-steampunk) of how this all works too.) SSE is old-school these days. Next.js and Create-React-App and all these things use websockets for their hot-module reloading. I doubt there's much _intrinsic_ advantage of SSE vs websockets for this little usecase, but the latter are much more useful and widespread so it's conceivable that Racket has nice websocket support that'll make the server component a breeze. Though unfamiliar with the Racket ecosystem, I'm happy to advise.

Another option would be for you to create a new project server (in a separate Racket package) and then in Pollen, we could add a new pollen/setup value that points raco pollen start ··· toward whatever project server one prefers. (This is similar to how @otherjoel achieved support for his external Pollen renderer within his beeswax project). No part of Pollen is a miracle of software engineering, but that is especially true of the project server, so I can’t recommend that you spend a lot of time learning its hacky habits.

Another option would be for you to create a new project server (in a separate Racket package) and then in Pollen, we could add a new `pollen/setup` value that points `raco pollen start ···` toward whatever project server one prefers. (This is similar to how @otherjoel achieved support for his external Pollen renderer within his `beeswax` project). No part of Pollen is a miracle of software engineering, but that is especially true of the project server, so I can’t recommend that you spend a lot of time learning its hacky habits.

I'm not a frontend guy myself, but I've used Websockets at work when we needed to trigger some event in the browser, so this will probably be my first avenue of investigation. I suspect it'll look a lot like your example, @fasiha, just replacing the EventSource with a WebSocket.

Matthew- hearing you loud and clear on the drawbacks of integrating this into the existing project server. If things get hairy, I'll be ready to cut my losses and start a standalone server implementation.

I'm not a frontend guy myself, but I've used Websockets at work when we needed to trigger some event in the browser, so this will probably be my first avenue of investigation. I suspect it'll look a lot like your example, @fasiha, just replacing the `EventSource` with a `WebSocket`. Matthew- hearing you loud and clear on the drawbacks of integrating this into the existing project server. If things get hairy, I'll be ready to cut my losses and start a standalone server implementation.

Labels Milestones

Feature request: server auto-refreshes page when save detected #35