|
|
# General Phase Levels
|
|
|
|
|
|
A _phase_ can be thought of as a way to separate computations in a
|
|
|
pipeline of processes where one produces code that is used by the next.
|
|
|
\(E.g., a pipeline that consists of a preprocessor process, a compiler,
|
|
|
and an assembler.\)
|
|
|
|
|
|
Imagine starting two Racket processes for this purpose. If you ignore
|
|
|
inter-process communication channels like sockets and files, the
|
|
|
processes will have no way to share anything other than the text that is
|
|
|
piped from the standard output of one process into the standard input of
|
|
|
the other. Similarly, Racket effectively allows multiple invocations of
|
|
|
a module to exist in the same process but separated by phase. Racket
|
|
|
enforces _separation_ of such phases, where different phases cannot
|
|
|
communicate in any way other than via the protocol of macro expansion,
|
|
|
where the output of one phases is the code used in the next.
|
|
|
|
|
|
## 1. Phases and Bindings
|
|
|
|
|
|
Every binding of an identifier exists in a particular phase. The link
|
|
|
between a binding and its phase is represented by an integer _phase
|
|
|
level_. Phase level 0 is the phase used for “plain” \(or “runtime”\)
|
|
|
definitions, so
|
|
|
|
|
|
`(define` `age` `5)`
|
|
|
|
|
|
adds a binding for `age` into phase level 0. The identifier `age` can
|
|
|
be defined at a higher phase level using `begin-for-syntax`:
|
|
|
|
|
|
```racket
|
|
|
(begin-for-syntax
|
|
|
(define age 5))
|
|
|
```
|
|
|
|
|
|
With a single `begin-for-syntax` wrapper, `age` is defined at phase
|
|
|
level 1. We can easily mix these two definitions in the same module or
|
|
|
in a top-level namespace, and there is no clash between the two `age`s
|
|
|
that are defined at different phase levels:
|
|
|
|
|
|
```racket
|
|
|
> (define age 3)
|
|
|
> (begin-for-syntax
|
|
|
(define age 9))
|
|
|
```
|
|
|
|
|
|
The `age` binding at phase level 0 has a value of 3, and the `age`
|
|
|
binding at phase level 1 has a value of 9.
|
|
|
|
|
|
Syntax objects capture binding information as a first-class value. Thus,
|
|
|
|
|
|
`#'age`
|
|
|
|
|
|
is a syntax object that represents the `age` binding—but since there are
|
|
|
two `age`s \(one at phase level 0 and one at phase level 1\), which one
|
|
|
does it capture? In fact, Racket imbues `#'age` with lexical
|
|
|
information for all phase levels, so the answer is that `#'age` captures
|
|
|
both.
|
|
|
|
|
|
The relevant binding of `age` captured by `#'age` is determined when
|
|
|
`#'age` is eventually used. As an example, we bind `#'age` to a pattern
|
|
|
variable so we can use it in a template, and then we `eval`uate the
|
|
|
template: We use `eval` here to demonstrate phases, but see \[missing\]
|
|
|
for caveats about `eval`.
|
|
|
|
|
|
```racket
|
|
|
> (eval (with-syntax ([age #'age])
|
|
|
#'(displayln age)))
|
|
|
3
|
|
|
```
|
|
|
|
|
|
The result is `3` because `age` is used at phase 0 level. We can try
|
|
|
again with the use of `age` inside `begin-for-syntax`:
|
|
|
|
|
|
```racket
|
|
|
> (eval (with-syntax ([age #'age])
|
|
|
#'(begin-for-syntax
|
|
|
(displayln age))))
|
|
|
9
|
|
|
```
|
|
|
|
|
|
In this case, the answer is `9`, because we are using `age` at phase
|
|
|
level 1 instead of 0 \(i.e., `begin-for-syntax` evaluates its
|
|
|
expressions at phase level 1\). So, you can see that we started with the
|
|
|
same syntax object, `#'age`, and we were able to use it in two different
|
|
|
ways: at phase level 0 and at phase level 1.
|
|
|
|
|
|
A syntax object has a lexical context from the moment it first exists. A
|
|
|
syntax object that is provided from a module retains its lexical
|
|
|
context, and so it references bindings in the context of its source
|
|
|
module, not the context of its use. The following example defines
|
|
|
`button` at phase level 0 and binds it to `0`, while `see-button` binds
|
|
|
the syntax object for `button` in module `a`:
|
|
|
|
|
|
```racket
|
|
|
> (module a racket
|
|
|
(define button 0)
|
|
|
(provide (for-syntax see-button))
|
|
|
; Why not (define see-button #'button)? We explain later.
|
|
|
(define-for-syntax see-button #'button))
|
|
|
> (module b racket
|
|
|
(require 'a)
|
|
|
(define button 8)
|
|
|
(define-syntax (m stx)
|
|
|
see-button)
|
|
|
(m))
|
|
|
> (require 'b)
|
|
|
0
|
|
|
```
|
|
|
|
|
|
The result of the `m` macro is the value of `see-button`, which is
|
|
|
`#'button` with the lexical context of the `a` module. Even though
|
|
|
there is another `button` in `b`, the second `button` will not confuse
|
|
|
Racket, because the lexical context of `#'button` \(the value bound to
|
|
|
`see-button`\) is `a`.
|
|
|
|
|
|
Note that `see-button` is bound at phase level 1 by virtue of defining
|
|
|
it with `define-for-syntax`. Phase level 1 is needed because `m` is a
|
|
|
macro, so its body executes at one phase higher than the context of its
|
|
|
definition. Since `m` is defined at phase level 0, its body is at phase
|
|
|
level 1, so any bindings referenced by the body must be at phase level
|
|
|
1.
|
|
|
|
|
|
## 2. Phases and Modules
|
|
|
|
|
|
A phase level is a module-relative concept. When importing from another
|
|
|
module via `require`, Racket lets us shift imported bindings to a phase
|
|
|
level that is different from the original one:
|
|
|
|
|
|
```racket
|
|
|
(require "a.rkt") ; import with no phase shift
|
|
|
(require (for-syntax "a.rkt")) ; shift phase by +1
|
|
|
(require (for-template "a.rkt")) ; shift phase by -1
|
|
|
(require (for-meta 5 "a.rkt")) ; shift phase by +5
|
|
|
```
|
|
|
|
|
|
That is, using `for-syntax` in `require` means that all of the bindings
|
|
|
from that module will have their phase levels increased by one. A
|
|
|
binding that is `define`d at phase level 0 and imported with
|
|
|
`for-syntax` becomes a phase-level 1 binding:
|
|
|
|
|
|
```racket
|
|
|
> (module c racket
|
|
|
(define x 0) ; defined at phase level 0
|
|
|
(provide x))
|
|
|
> (module d racket
|
|
|
(require (for-syntax 'c))
|
|
|
; has a binding at phase level 1, not 0:
|
|
|
#'x)
|
|
|
```
|
|
|
|
|
|
Let’s see what happens if we try to create a binding for the `#'button`
|
|
|
syntax object at phase level 0:
|
|
|
|
|
|
```racket
|
|
|
> (define button 0)
|
|
|
> (define see-button #'button)
|
|
|
```
|
|
|
|
|
|
Now both `button` and `see-button` are defined at phase 0. The lexical
|
|
|
context of `#'button` will know that there is a binding for `button` at
|
|
|
phase 0. In fact, it seems like things are working just fine if we try
|
|
|
to `eval` `see-button`:
|
|
|
|
|
|
```racket
|
|
|
> (eval see-button)
|
|
|
0
|
|
|
```
|
|
|
|
|
|
Now, let’s use `see-button` in a macro:
|
|
|
|
|
|
```racket
|
|
|
> (define-syntax (m stx)
|
|
|
see-button)
|
|
|
> (m)
|
|
|
see-button: undefined;
|
|
|
cannot reference an identifier before its definition
|
|
|
in module: top-level
|
|
|
```
|
|
|
|
|
|
Clearly, `see-button` is not defined at phase level 1, so we cannot
|
|
|
refer to it inside the macro body. Let’s try to use `see-button` in
|
|
|
another module by putting the button definitions in a module and
|
|
|
importing it at phase level 1. Then, we will get `see-button` at phase
|
|
|
level 1:
|
|
|
|
|
|
```racket
|
|
|
> (module a racket
|
|
|
(define button 0)
|
|
|
(define see-button #'button)
|
|
|
(provide see-button))
|
|
|
> (module b racket
|
|
|
(require (for-syntax 'a)) ; gets see-button at phase level 1
|
|
|
(define-syntax (m stx)
|
|
|
see-button)
|
|
|
(m))
|
|
|
eval:1:0: button: unbound identifier;
|
|
|
also, no #%top syntax transformer is bound
|
|
|
in: button
|
|
|
```
|
|
|
|
|
|
Racket says that `button` is unbound now! When `a` is imported at phase
|
|
|
level 1, we have the following bindings:
|
|
|
|
|
|
```racket
|
|
|
button at phase level 1
|
|
|
see-button at phase level 1
|
|
|
```
|
|
|
|
|
|
So the macro `m` can see a binding for `see-button` at phase level 1 and
|
|
|
will return the `#'button` syntax object, which refers to `button`
|
|
|
binding at phase level 1. But the use of `m` is at phase level 0, and
|
|
|
there is no `button` at phase level 0 in `b`. That is why `see-button`
|
|
|
needs to be bound at phase level 1, as in the original `a`. In the
|
|
|
original `b`, then, we have the following bindings:
|
|
|
|
|
|
```racket
|
|
|
button at phase level 0
|
|
|
see-button at phase level 1
|
|
|
```
|
|
|
|
|
|
In this scenario, we can use `see-button` in the macro, since
|
|
|
`see-button` is bound at phase level 1. When the macro expands, it will
|
|
|
refer to a `button` binding at phase level 0.
|
|
|
|
|
|
Defining `see-button` with `(define see-button #'button)` isn’t
|
|
|
inherently wrong; it depends on how we intend to use `see-button`. For
|
|
|
example, we can arrange for `m` to sensibly use `see-button` because it
|
|
|
puts it in a phase level 1 context using `begin-for-syntax`:
|
|
|
|
|
|
```racket
|
|
|
> (module a racket
|
|
|
(define button 0)
|
|
|
(define see-button #'button)
|
|
|
(provide see-button))
|
|
|
> (module b racket
|
|
|
(require (for-syntax 'a))
|
|
|
(define-syntax (m stx)
|
|
|
(with-syntax ([x see-button])
|
|
|
#'(begin-for-syntax
|
|
|
(displayln x))))
|
|
|
(m))
|
|
|
0
|
|
|
```
|
|
|
|
|
|
In this case, module `b` has both `button` and `see-button` bound at
|
|
|
phase level 1. The expansion of the macro is
|
|
|
|
|
|
```racket
|
|
|
(begin-for-syntax
|
|
|
(displayln button))
|
|
|
```
|
|
|
|
|
|
which works, because `button` is bound at phase level 1.
|
|
|
|
|
|
Now, you might try to cheat the phase system by importing `a` at both
|
|
|
phase level 0 and phase level 1. Then you would have the following
|
|
|
bindings
|
|
|
|
|
|
```racket
|
|
|
button at phase level 0
|
|
|
see-button at phase level 0
|
|
|
button at phase level 1
|
|
|
see-button at phase level 1
|
|
|
```
|
|
|
|
|
|
You might expect now that `see-button` in a macro would work, but it
|
|
|
doesn’t:
|
|
|
|
|
|
```racket
|
|
|
> (module a racket
|
|
|
(define button 0)
|
|
|
(define see-button #'button)
|
|
|
(provide see-button))
|
|
|
> (module b racket
|
|
|
(require 'a
|
|
|
(for-syntax 'a))
|
|
|
(define-syntax (m stx)
|
|
|
see-button)
|
|
|
(m))
|
|
|
eval:1:0: button: unbound identifier;
|
|
|
also, no #%top syntax transformer is bound
|
|
|
in: button
|
|
|
```
|
|
|
|
|
|
The `see-button` inside macro `m` comes from the `(for-syntax 'a)`
|
|
|
import. For macro `m` to work, it needs to have `button` bound at phase
|
|
|
0. That binding exists—it’s implied by `(require 'a)`. However,
|
|
|
`(require 'a)` and `(require (for-syntax 'a))` are _different
|
|
|
instantiations_ of the same module. The `see-button` at phase 1 only
|
|
|
refers to the `button` at phase 1, not the `button` bound at phase 0
|
|
|
from a different instantiation—even from the same source module.
|
|
|
|
|
|
This kind of phase-level mismatch between instantiations can be repaired
|
|
|
with `syntax-shift-phase-level`. Recall that a syntax object like
|
|
|
`#'button` captures lexical information at _all_ phase levels. The
|
|
|
problem here is that `see-button` is invoked at phase 1, but needs to
|
|
|
return a syntax object that can be evaluated at phase 0. By default,
|
|
|
`see-button` is bound to `#'button` at the same phase level. But with
|
|
|
`syntax-shift-phase-level`, we can make `see-button` refer to `#'button`
|
|
|
at a different relative phase level. In this case, we use a phase shift
|
|
|
of `-1` to make `see-button` at phase 1 refer to `#'button` at phase 0.
|
|
|
\(Because the phase shift happens at every level, it will also make
|
|
|
`see-button` at phase 0 refer to `#'button` at phase -1.\)
|
|
|
|
|
|
Note that `syntax-shift-phase-level` merely creates a reference across
|
|
|
phases. To make that reference work, we still need to instantiate our
|
|
|
module at both phases so the reference and its target have their
|
|
|
bindings available. Thus, in module `'b`, we still import module `'a` at
|
|
|
both phase 0 and phase 1—using `(require 'a (for-syntax 'a))`—so we have
|
|
|
a phase-1 binding for `see-button` and a phase-0 binding for `button`.
|
|
|
Now macro `m` will work.
|
|
|
|
|
|
```racket
|
|
|
> (module a racket
|
|
|
(define button 0)
|
|
|
(define see-button (syntax-shift-phase-level #'button -1))
|
|
|
(provide see-button))
|
|
|
> (module b racket
|
|
|
(require 'a (for-syntax 'a))
|
|
|
(define-syntax (m stx)
|
|
|
see-button)
|
|
|
(m))
|
|
|
> (require 'b)
|
|
|
0
|
|
|
```
|
|
|
|
|
|
By the way, what happens to the `see-button` that’s bound at phase 0?
|
|
|
Its `#'button` binding has likewise been shifted, but to phase -1. Since
|
|
|
`button` itself isn’t bound at phase -1, if we try to evaluate
|
|
|
`see-button` at phase 0, we get an error. In other words, we haven’t
|
|
|
permanently cured our mismatch problem—we’ve just shifted it to a less
|
|
|
bothersome location.
|
|
|
|
|
|
```racket
|
|
|
> (module a racket
|
|
|
(define button 0)
|
|
|
(define see-button (syntax-shift-phase-level #'button -1))
|
|
|
(provide see-button))
|
|
|
> (module b racket
|
|
|
(require 'a (for-syntax 'a))
|
|
|
(define-syntax (m stx)
|
|
|
see-button)
|
|
|
(m))
|
|
|
> (module b2 racket
|
|
|
(require 'a)
|
|
|
(eval see-button))
|
|
|
> (require 'b2)
|
|
|
button: undefined;
|
|
|
cannot reference an identifier before its definition
|
|
|
in module: top-level
|
|
|
```
|
|
|
|
|
|
Mismatches like the one above can also arise when a macro tries to match
|
|
|
literal bindings—using `syntax-case` or `syntax-parse`.
|
|
|
|
|
|
```racket
|
|
|
> (module x racket
|
|
|
(require (for-syntax syntax/parse)
|
|
|
(for-template racket/base))
|
|
|
|
|
|
(provide (all-defined-out))
|
|
|
|
|
|
(define button 0)
|
|
|
(define (make) #'button)
|
|
|
(define-syntax (process stx)
|
|
|
(define-literal-set locals (button))
|
|
|
(syntax-parse stx
|
|
|
[(_ (n (~literal button))) #'#''ok])))
|
|
|
> (module y racket
|
|
|
(require (for-meta 1 'x)
|
|
|
(for-meta 2 'x racket/base))
|
|
|
|
|
|
(begin-for-syntax
|
|
|
(define-syntax (m stx)
|
|
|
(with-syntax ([out (make)])
|
|
|
#'(process (0 out)))))
|
|
|
|
|
|
(define-syntax (p stx)
|
|
|
(m))
|
|
|
|
|
|
(p))
|
|
|
eval:2.0: process: expected the identifier `button'
|
|
|
at: button
|
|
|
in: (process (0 button))
|
|
|
```
|
|
|
|
|
|
In this example, `make` is being used in `y` at phase level 2, and it
|
|
|
returns the `#'button` syntax object—which refers to `button` bound at
|
|
|
phase level 0 inside `x` and at phase level 2 in `y` from `(for-meta 2
|
|
|
'x)`. The `process` macro is imported at phase level 1 from `(for-meta
|
|
|
1 'x)`, and it knows that `button` should be bound at phase level 1.
|
|
|
When the `syntax-parse` is executed inside `process`, it is looking for
|
|
|
`button` bound at phase level 1 but it sees only a phase level 2 binding
|
|
|
and doesn’t match.
|
|
|
|
|
|
To fix the example, we can provide `make` at phase level 1 relative to
|
|
|
`x`, and then we import it at phase level 1 in `y`:
|
|
|
|
|
|
```racket
|
|
|
> (module x racket
|
|
|
(require (for-syntax syntax/parse)
|
|
|
(for-template racket/base))
|
|
|
|
|
|
(provide (all-defined-out))
|
|
|
|
|
|
(define button 0)
|
|
|
|
|
|
(provide (for-syntax make))
|
|
|
(define-for-syntax (make) #'button)
|
|
|
(define-syntax (process stx)
|
|
|
(define-literal-set locals (button))
|
|
|
(syntax-parse stx
|
|
|
[(_ (n (~literal button))) #'#''ok])))
|
|
|
> (module y racket
|
|
|
(require (for-meta 1 'x)
|
|
|
(for-meta 2 racket/base))
|
|
|
|
|
|
(begin-for-syntax
|
|
|
(define-syntax (m stx)
|
|
|
(with-syntax ([out (make)])
|
|
|
#'(process (0 out)))))
|
|
|
|
|
|
(define-syntax (p stx)
|
|
|
(m))
|
|
|
|
|
|
(p))
|
|
|
> (require 'y)
|
|
|
'ok
|
|
|
```
|