You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
typesetting/quad/qtest/mds/languages.md

45 KiB

Creating Languages

The macro facilities defined in the preceding chapter let a programmer define syntactic extensions to a language, but a macro is limited in two ways:

  • a macro cannot restrict the syntax available in its context or change the meaning of surrounding forms; and

  • a macro can extend the syntax of a language only within the parameters of the languages lexical conventions, such as using parentheses to group the macro name with its subforms and using the core syntax of identifiers, keywords, and literals.

+The distinction between the reader and expander layer is introduced in

missing

That is, a macro can only extend a language, and it can do so only at the expander layer. Racket offers additional facilities for defining a starting point of the expander layer, for extending the reader layer, for defining the starting point of the reader layer, and for packaging a reader and expander starting point into a conveniently named language.

1 Module Languages                            
  1.1 Implicit Form Bindings                  
  1.2 Using `#lang s-exp`                     
                                              
2 Reader Extensions                           
  2.1 Source Locations                        
  2.2 Readtables                              
                                              
3 Defining new `#lang` Languages              
  3.1 Designating a `#lang` Language          
  3.2 Using `#lang reader`                    
  3.3 Using `#lang s-exp syntax/module-reader`
  3.4 Installing a Language                   
  3.5 Source-Handling Configuration           
  3.6 Module-Handling Configuration           

1. Module Languages

When using the longhand module form for writing modules, the module path that is specified after the new modules name provides the initial imports for the module. Since the initial-import module determines even the most basic bindings that are available in a modules body, such as require, the initial import can be called a module language.

The most common module languages are racket or racket/base, but you can define your own module language by defining a suitable module. For example, using provide subforms like all-from-out, except-out, and rename-out, you can add, remove, or rename bindings from racket to produce a module language that is a variant of racket:

+[missing] introduces the longhand module form.

> (module raquet racket                                    
    (provide (except-out (all-from-out racket) lambda)     
             (rename-out [lambda function])))              
> (module score 'raquet                                    
    (map (function (points) (case points                   
                             [(0) "love"] [(1) "fifteen"]  
                             [(2) "thirty"] [(3) "forty"]))
         (list 0 2)))                                      
> (require 'score)                                         
'("love" "thirty")                                         

1.1. Implicit Form Bindings

If you try to remove too much from racket in defining your own module language, then the resulting module will no longer work right as a module language:

> (module just-lambda racket                               
    (provide lambda))                                      
> (module identity 'just-lambda                            
    (lambda (x) x))                                        
eval:2:0: module: no #%module-begin binding in the module's
language                                                   
  in: (module identity (quote just-lambda) (lambda (x) x)) 

The #%module-begin form is an implicit form that wraps the body of a module. It must be provided by a module that is to be used as module language:

> (module just-lambda racket        
    (provide lambda #%module-begin))
> (module identity 'just-lambda     
    (lambda (x) x))                 
> (require 'identity)               
#<procedure>                        

The other implicit forms provided by racket/base are #%app for function calls, #%datum for literals, and #%top for identifiers that have no binding:

> (module just-lambda racket        
    (provide lambda #%module-begin  
             ; ten needs these, too:
             #%app #%datum))        
> (module ten 'just-lambda          
    ((lambda (x) x) 10))            
> (require 'ten)                    
10                                  

Implicit forms such as #%app can be used explicitly in a module, but they exist mainly to allow a module language to restrict or change the meaning of implicit uses. For example, a lambda-calculus module language might restrict functions to a single argument, restrict function calls to supply a single argument, restrict the module body to a single expression, disallow literals, and treat unbound identifiers as uninterpreted symbols:

> (module lambda-calculus racket                             
    (provide (rename-out [1-arg-lambda lambda]               
                         [1-arg-app #%app]                   
                         [1-form-module-begin #%module-begin]
                         [no-literals #%datum]               
                         [unbound-as-quoted #%top]))         
    (define-syntax-rule (1-arg-lambda (x) expr)              
      (lambda (x) expr))                                     
    (define-syntax-rule (1-arg-app e1 e2)                    
      (#%app e1 e2))                                         
    (define-syntax-rule (1-form-module-begin e)              
      (#%module-begin e))                                    
    (define-syntax (no-literals stx)                         
      (raise-syntax-error #f "no" stx))                      
    (define-syntax-rule (unbound-as-quoted . id)             
      'id))                                                  
> (module ok 'lambda-calculus                                
    ((lambda (x) (x z))                                      
     (lambda (y) y)))                                        
> (require 'ok)                                              
'z                                                           
> (module not-ok 'lambda-calculus                            
    (lambda (x y) x))                                        
eval:4:0: lambda: use does not match pattern: (lambda (x)    
expr)                                                        
  in: (lambda (x y) x)                                       
> (module not-ok 'lambda-calculus                            
    (lambda (x) x)                                           
    (lambda (y) (y y)))                                      
eval:5:0: #%module-begin: use does not match pattern:        
(#%module-begin e)                                           
  in: (#%module-begin (lambda (x) x) (lambda (y) (y y)))     
> (module not-ok 'lambda-calculus                            
    (lambda (x) (x x x)))                                    
eval:6:0: #%app: use does not match pattern: (#%app e1 e2)   
  in: (#%app x x x)                                          
> (module not-ok 'lambda-calculus                            
    10)                                                      
eval:7:0: #%datum: no                                        
  in: (#%datum . 10)                                         

Module languages rarely redefine #%app, #%datum, and #%top, but redefining #%module-begin is more frequently useful. For example, when using modules to construct descriptions of HTML pages where a description is exported from the module as page, an alternate #%module-begin can help eliminate provide and quasiquoting boilerplate, as in "html.rkt":

"html.rkt"

#lang racket                                          
(require racket/date)                                 
                                                      
(provide (except-out (all-from-out racket)            
                     #%module-begin)                  
         (rename-out [module-begin #%module-begin])   
         now)                                         
                                                      
(define-syntax-rule (module-begin expr ...)           
  (#%module-begin                                     
   (define page `(html expr ...))                     
   (provide page)))                                   
                                                      
(define (now)                                         
  (parameterize ([date-display-format 'iso-8601])     
    (date->string (seconds->date (current-seconds)))))

Using the "html.rkt" module language, a simple web page can be described without having to explicitly define or export page and starting in quasiquoted mode instead of expression mode:

> (module lady-with-the-spinning-head "html.rkt"                
    (title "Queen of Diamonds")                                 
    (p "Updated: " ,(now)))                                     
> (require 'lady-with-the-spinning-head)                        
> page                                                          
'(html (title "Queen of Diamonds") (p "Updated: " "2019-01-21"))

1.2. Using #lang`` ``s-exp

Implementing a language at the level of #lang is more complex than declaring a single module, because #lang lets programmers control several different facets of a language. The s-exp language, however, acts as a kind of meta-language for using a module language with the #lang shorthand:

#lang s-exp module-name
form ...               

is the same as

(module name module-name
  form ...)             

where name is derived from the source file containing the #lang program. The name s-exp is short for “S-expression,” which is a traditional name for Rackets reader-level lexical conventions: parentheses, identifiers, numbers, double-quoted strings with certain backslash escapes, and so on.

Using #lang s-exp, the lady-with-the-spinning-head example from before can be written more compactly as:

#lang s-exp "html.rkt"     
                           
(title "Queen of Diamonds")
(p "Updated: " ,(now))     

Later in this guide, Defining new #lang Languages explains how to define your own #lang language, but first we explain how you can write reader-level extensions to Racket.

2. Reader Extensions

+[missing] in [missing] provides more on reader extensions.

The reader layer of the Racket language can be extended through the #reader form. A reader extension is implemented as a module that is named after #reader. The module exports functions that parse raw characters into a form to be consumed by the expander layer.

The syntax of #reader is

#reader >module-path< >reader-specific<

where >module-path< names a module that provides read and read-syntax functions. The >reader-specific< part is a sequence of characters that is parsed as determined by the read and read-syntax functions from >module-path<.

For example, suppose that file "five.rkt" contains

"five.rkt"

#lang racket/base                                      
                                                       
(provide read read-syntax)                             
                                                       
(define (read in) (list (read-string 5 in)))           
(define (read-syntax src in) (list (read-string 5 in)))

Then, the program

#lang racket/base             
                              
'(1 #reader"five.rkt"234567 8)

is equivalent to

#lang racket/base 
                  
'(1 ("23456") 7 8)

because the read and read-syntax functions of "five.rkt" both read five characters from the input stream and put them into a string and then a list. The reader functions from "five.rkt" are not obliged to follow Racket lexical conventions and treat the continuous sequence 234567 as a single number. Since only the 23456 part is consumed by read or read-syntax, the 7 remains to be parsed in the usual Racket way. Similarly, the reader functions from "five.rkt" are not obliged to ignore whitespace, and

#lang racket/base              
                               
'(1 #reader"five.rkt" 234567 8)

is equivalent to

#lang racket/base  
                   
'(1 (" 2345") 67 8)

since the first character immediately after "five.rkt" is a space.

A #reader form can be used in the REPL, too:

> '#reader"five.rkt"abcde
'("abcde")               

2.1. Source Locations

The difference between read and read-syntax is that read is meant to be used for data while read-syntax is meant to be used to parse programs. More precisely, the read function will be used when the enclosing stream is being parsed by the Racket read, and read-syntax is used when the enclosing stream is being parsed by the Racket read-syntax function. Nothing requires read and read-syntax to parse input in the same way, but making them different would confuse programmers and tools.

The read-syntax function can return the same kind of value as read, but it should normally return a syntax object that connects the parsed expression with source locations. Unlike the "five.rkt" example, the read-syntax function is typically implemented directly to produce syntax objects, and then read can use read-syntax and strip away syntax object wrappers to produce a raw result.

The following "arith.rkt" module implements a reader to parse simple infix arithmetic expressions into Racket forms. For example, 1*2+3 parses into the Racket form (+ (* 1 2) 3). The supported operators are +, -, *, and /, while operands can be unsigned integers or single-letter variables. The implementation uses port-next-location to obtain the current source location, and it uses datum->syntax to turn raw values into syntax objects.

"arith.rkt"

#lang racket                                                
(require syntax/readerr)                                    
                                                            
(provide read read-syntax)                                  
                                                            
(define (read in)                                           
  (syntax->datum (read-syntax #f in)))                      
                                                            
(define (read-syntax src in)                                
  (skip-whitespace in)                                      
  (read-arith src in))                                      
                                                            
(define (skip-whitespace in)                                
  (regexp-match #px"^\\s*" in))                             
                                                            
(define (read-arith src in)                                 
  (define-values (line col pos) (port-next-location in))    
  (define expr-match                                        
    (regexp-match                                           
     ; Match an operand followed by any number of           
     ; operatoroperand sequences, and prohibit an          
     ; additional operator from following immediately:      
     #px"^([a-z]|[0-9]+)(?:[-+*/]([a-z]|[0-9]+))*(?![-+*/])"
     in))                                                   
                                                            
  (define (to-syntax v delta span-str)                      
    (datum->syntax #f v (make-srcloc delta span-str)))      
  (define (make-srcloc delta span-str)                      
    (and line                                               
         (vector src line (+ col delta) (+ pos delta)       
                 (string-length span-str))))                
                                                            
  (define (parse-expr s delta)                              
    (match (or (regexp-match #rx"^(.*?)([+-])(.*)$" s)      
               (regexp-match #rx"^(.*?)([*/])(.*)$" s))     
      [(list _ a-str op-str b-str)                          
       (define a-len (string-length a-str))                 
       (define a (parse-expr a-str delta))                  
       (define b (parse-expr b-str (+ delta 1 a-len)))      
       (define op (to-syntax (string->symbol op-str)        
                             (+ delta a-len) op-str))       
       (to-syntax (list op a b) delta s)]                   
      [_ (to-syntax (or (string->number s)                  
                        (string->symbol s))                 
                    delta s)]))                             
                                                            
  (unless expr-match                                        
    (raise-read-error "bad arithmetic syntax"               
                      src line col pos                      
                      (and pos (- (file-position in) pos))))
  (parse-expr (bytes->string/utf-8 (car expr-match)) 0))    

If the "arith.rkt" reader is used in an expression position, then its parse result will be treated as a Racket expression. If it is used in a quoted form, however, then it just produces a number or a list:

> #reader"arith.rkt" 1*2+3 
5                          
> '#reader"arith.rkt" 1*2+3
'(+ (* 1 2) 3)             

The "arith.rkt" reader could also be used in positions that make no sense. Since the read-syntax implementation tracks source locations, syntax errors can at least refer to parts of the input in terms of their original locations at the beginning of the error message:

> (let #reader"arith.rkt" 1*2+3 8)                          
repl:1:27: let: bad syntax (not an identifier and expression
for a binding)                                              
  at: +                                                     
  in: (let (+ (* 1 2) 3) 8)                                 

2.2. Readtables

A reader extensions ability to parse input characters in an arbitrary way can be powerful, but many cases of lexical extension call for a less general but more composable approach. In much the same way that the expander level of Racket syntax can be extended through macros, the reader level of Racket syntax can be composably extended through a readtable.

The Racket reader is a recursive-descent parser, and the readtable maps characters to parsing handlers. For example, the default readtable maps ( to a handler that recursively parses subforms until it finds a ). The current-readtable parameter determines the readtable that is used by read or read-syntax. Rather than parsing raw characters directly, a reader extension can install an extended readtable and then chain to read or read-syntax.

+See [missing] for an introduction to parameters.

The make-readtable function constructs a new readtable as an extension of an existing one. It accepts a sequence of specifications in terms of a character, a type of mapping for the character, and (for certain types of mappings) a parsing procedure. For example, to extend the readtable so that $ can be used to start and end infix expressions, implement a read-dollar function and use:

(make-readtable (current-readtable)                
                #\$ 'terminating-macro read-dollar)

The protocol for read-dollar requires the function to accept different numbers of arguments depending on whether it is being used in read or read-syntax mode. In read mode, the parser function is given two arguments: the character that triggered the parser function and the input port that is being read. In read-syntax mode, the function must accept four additional arguments that provide the source location of the character.

The following "dollar.rkt" module defines a read-dollar function in terms of the read and read-syntax functions provided by "arith.rkt", and it puts read-dollar together with new read and read-syntax functions that install the readtable and chain to Rackets read or read-syntax:

"dollar.rkt"

#lang racket                                            
(require syntax/readerr                                 
         (prefix-in arith: "arith.rkt"))                
                                                        
(provide (rename-out [$-read read]                      
                     [$-read-syntax read-syntax]))      
                                                        
(define ($-read in)                                     
  (parameterize ([current-readtable (make-$-readtable)])
    (read in)))                                         
                                                        
(define ($-read-syntax src in)                          
  (parameterize ([current-readtable (make-$-readtable)])
    (read-syntax src in)))                              
                                                        
(define (make-$-readtable)                              
  (make-readtable (current-readtable)                   
                  #\$ 'terminating-macro read-dollar))  
                                                        
(define read-dollar                                     
  (case-lambda                                          
   [(ch in)                                             
    (check-$-after (arith:read in) in (object-name in))]
   [(ch in src line col pos)                            
    (check-$-after (arith:read-syntax src in) in src)]))
                                                        
(define (check-$-after val in src)                      
  (regexp-match #px"^\\s*" in) ; skip whitespace        
  (let ([ch (peek-char in)])                            
    (unless (equal? ch #\$) (bad-ending ch src in))     
    (read-char in))                                     
  val)                                                  
                                                        
(define (bad-ending ch src in)                          
  (let-values ([(line col pos) (port-next-location in)])
    ((if (eof-object? ch)                               
         raise-read-error                               
         raise-read-eof-error)                          
     "expected a closing `$'"                           
     src line col pos                                   
     (if (eof-object? ch) 0 1))))                       

With this reader extension, a single #reader can be used at the beginning of an expression to enable multiple uses of $ that switch to infix arithmetic:

> #reader"dollar.rkt" (let ([a $1*2+3$] [b $5/6$]) $a+b$)
35/6                                                     

3. Defining new #lang Languages

When loading a module as a source program that starts

#lang language

the language determines the way that the rest of the module is parsed at the reader level. The reader-level parse must produce a module form as a syntax object. As always, the second sub-form after module specifies the module language that controls the meaning of the modules body forms. Thus, a language specified after #lang controls both the reader-level and expander-level parsing of a module.

3.1 Designating a `#lang` Language          
3.2 Using `#lang reader`                    
3.3 Using `#lang s-exp syntax/module-reader`
3.4 Installing a Language                   
3.5 Source-Handling Configuration           
3.6 Module-Handling Configuration           

3.1. Designating a #lang Language

The syntax of a language intentionally overlaps with the syntax of a module path as used in require or as a module language, so that names like racket, racket/base, slideshow, or scribble/manual can be used both as #lang languages and as module paths.

At the same time, the syntax of language is far more restricted than a module path, because only a-z, A-Z, 0-9, / (not at the start or end), _, -, and + are allowed in a language name. These restrictions keep the syntax of #lang as simple as possible. Keeping the syntax of #lang simple, in turn, is important because the syntax is inherently inflexible and non-extensible; the #lang protocol allows a language to refine and define syntax in a practically unconstrained way, but the #lang protocol itself must remain fixed so that various different tools can “boot” into the extended world.

Fortunately, the #lang protocol provides a natural way to refer to languages in ways other than the rigid language syntax: by defining a language that implements its own nested protocol. We have already seen one example in Using `#lang s-exp`: the s-exp language allows a programmer to specify a module language using the general module path syntax. Meanwhile, s-exp takes care of the reader-level responsibilities of a #lang language.

Unlike racket, s-exp cannot be used as a module path with require. Although the syntax of language for #lang overlaps with the syntax of module paths, a language is not used directly as a module path. Instead, a language obtains a module path by trying two locations: first, it looks for a reader submodule of the main module for language. If this is not a valid module path, then language is suffixed with /lang/reader. (If neither is a valid module path, an error is raised.) The resulting module supplies read and read-syntax functions using a protocol that is similar to the one for #reader.

+Reader Extensions introduces #reader.

A consequence of the way that a #lang language is turned into a module path is that the language must be installed in a collection, similar to the way that "racket" or "slideshow" are collections that are distributed with Racket. Again, however, theres an escape from this restriction: the reader language lets you specify a reader-level implementation of a language using a general module path.

3.2. Using #lang`` ``reader

The reader language for #lang is similar to s-exp, in that it acts as a kind of meta-language. Whereas s-exp lets a programmer specify a module language at the expander layer of parsing, reader lets a programmer specify a language at the reader level.

A #lang reader must be followed by a module path, and the specified module must provide two functions: read and read-syntax. The protocol is the same as for a #reader implementation, but for #lang, the read and read-syntax functions must produce a module form that is based on the rest of the input file for the module.

The following "literal.rkt" module implements a language that treats its entire body as literal text and exports the text as a data string:

"literal.rkt"

#lang racket                                            
(require syntax/strip-context)                          
                                                        
(provide (rename-out [literal-read read]                
                     [literal-read-syntax read-syntax]))
                                                        
(define (literal-read in)                               
  (syntax->datum                                        
   (literal-read-syntax #f in)))                        
                                                        
(define (literal-read-syntax src in)                    
  (with-syntax ([str (port->string in)])                
    (strip-context                                      
     #'(module anything racket                          
         (provide data)                                 
         (define data 'str)))))                         

The "literal.rkt" language uses strip-context on the generated module expression, because a read-syntax function should return a syntax object with no lexical context. Also, the "literal.rkt" language creates a module named anything, which is an arbitrary choice; the language is intended to be used in a file, and the longhand module name is ignored when it appears in a required file.

The "literal.rkt" language can be used in a module "tuvalu.rkt":

"tuvalu.rkt"

#lang reader "literal.rkt"
Technology!               
System!                   
Perfect!                  

Importing "tuvalu.rkt" binds data to a string version of the module content:

> (require "tuvalu.rkt")            
> data                              
"\nTechnology!\nSystem!\nPerfect!\n"

3.3. Using #lang`` ``s-exp`` ``syntax/module-reader

Parsing a module body is usually not as trivial as in "literal.rkt". A more typical module parser must iterate to parse multiple forms for a module body. A language is also more likely to extend Racket syntax—perhaps through a readtable—instead of replacing Racket syntax completely.

The syntax/module-reader module language abstracts over common parts of a language implementation to simplify the creation of new languages. In its most basic form, a language implemented with syntax/module-reader simply specifies the module language to be used for the language, in which case the reader layer of the language is the same as Racket. For example, with

"raquet-mlang.rkt"

#lang racket                                      
(provide (except-out (all-from-out racket) lambda)
         (rename-out [lambda function]))          

and

"raquet.rkt"

#lang s-exp syntax/module-reader
"raquet-mlang.rkt"              

then

#lang reader "raquet.rkt"         
(define identity (function (x) x))
(provide identity)                

implements and exports the identity function, since "raquet-mlang.rkt" exports lambda as function.

The syntax/module-reader language accepts many optional specifications to adjust other features of the language. For example, an alternate read and read-syntax for parsing the language can be specified with #:read and #:read-syntax, respectively. The following "dollar-racket.rkt" language uses "dollar.rkt" see Readtables to build a language that is like racket but with a $ escape to simple infix arithmetic:

"dollar-racket.rkt"

#lang s-exp syntax/module-reader     
racket                               
#:read $-read                        
#:read-syntax $-read-syntax          
                                     
(require (prefix-in $- "dollar.rkt"))

The require form appears at the end of the module, because all of the keyword-tagged optional specifications for syntax/module-reader must appear before any helper imports or definitions.

The following module uses "dollar-racket.rkt" to implement a cost function using a $ escape:

"store.rkt"

#lang reader "dollar-racket.rkt"        
                                        
(provide cost)                          
                                        
; Cost of n' $1 rackets with 7% sales  
; tax and shipping-and-handling fee h':
(define (cost n h)                      
  $n*107/100+h$)                        

3.4. Installing a Language

So far, we have used the reader meta-language to access languages like "literal.rkt" and "dollar-racket.rkt". If you want to use something like #lang literal directly, then you must move "literal.rkt" into a Racket collection named "literal" see also \[missing\]. Specifically, move "literal.rkt" to a reader submodule of "literal/main.rkt" for any directory name "literal", like so:

"literal/main.rkt"

#lang racket                                              
                                                          
(module reader racket                                     
  (require syntax/strip-context)                          
                                                          
  (provide (rename-out [literal-read read]                
                       [literal-read-syntax read-syntax]))
                                                          
  (define (literal-read in)                               
    (syntax->datum                                        
     (literal-read-syntax #f in)))                        
                                                          
  (define (literal-read-syntax src in)                    
    (with-syntax ([str (port->string in)])                
      (strip-context                                      
       #'(module anything racket                          
           (provide data)                                 
           (define data 'str))))))                        

Then, install the "literal" directory as a package:

  cd /path/to/literal ; raco pkg install

After moving the file and installing the package, you can use literal directly after #lang:

#lang literal
Technology!  
System!      
Perfect!     

See [missing] for more information on using raco.

You can also make your language available for others to install by using the Racket package manager see \[missing\]. After you create a "literal" package and register it with the Racket package catalog see \[missing\], others can install it using raco pkg:

  raco pkg install literal

Once installed, others can invoke the language the same way: by using #lang literal at the top of a source file.

If you use a public source repository e.g., GitHub, you can link your package to the source. As you improve the package, others can update their version using raco pkg:

  raco pkg update literal

See [missing] for more information about the Racket package manager.

3.5. Source-Handling Configuration

The Racket distribution includes a Scribble language for writing prose documents, where Scribble extends the normal Racket to better support text. Here is an example Scribble document:

#lang scribble/base

@(define (get-name) "Self-Describing Document")

@title[(get-name)]

The title of this document is “@(get-name).”

If you put that program in DrRackets definitions area and click Run, then nothing much appears to happen. The scribble/base language just binds and exports doc as a description of a document, similar to the way that "literal.rkt" exports a string as data.

Simply opening a module with the language scribble/base in DrRacket, however, causes a Scribble HTML button to appear. Furthermore, DrRacket knows how to colorize Scribble syntax by coloring green those parts of the document that correspond to literal text. The language name scribble/base is not hard-wired into DrRacket. Instead, the implementation of the scribble/base language provides button and syntax-coloring information in response to a query from DrRacket.

If you have installed the literal language as described in Installing a Language, then you can adjust "literal/main.rkt" so that DrRacket treats the content of a module in the literal language as plain text instead of erroneously as Racket syntax:

"literal/main.rkt"

#lang racket                                             
                                                         
(module reader racket                                    
  (require syntax/strip-context)                         
                                                         
  (provide (rename-out [literal-read read]               
                       [literal-read-syntax read-syntax])
           get-info)                                     
                                                         
  (define (literal-read in)                              
    (syntax->datum                                       
     (literal-read-syntax #f in)))                       
                                                         
  (define (literal-read-syntax src in)                   
    (with-syntax ([str (port->string in)])               
      (strip-context                                     
       #'(module anything racket                         
           (provide data)                                
           (define data 'str)))))                        
                                                         
  (define (get-info in mod line col pos)                 
    (lambda (key default)                                
      (case key                                          
        [(color-lexer)                                   
         (dynamic-require 'syntax-color/default-lexer    
                          'default-lexer)]               
        [else default]))))                               

This revised literal implementation provides a get-info function. The get-info function is called by read-language (which DrRacket calls) with the source input stream and location information, in case query results should depend on the content of the module after the language name which is not the case for `literal`. The result of get-info is a function of two arguments. The first argument is always a symbol, indicating the kind of information that a tool requests from the language; the second argument is the default result to be returned if the language does not recognize the query or has no information for it.

After DrRacket obtains the result of get-info for a language, it calls the function with a 'color-lexer query; the result should be a function that implements syntax-coloring parsing on an input stream. For literal, the syntax-color/default-lexer module provides a default-lexer syntax-coloring parser that is suitable for plain text, so literal loads and returns that parser in response to a 'color-lexer query.

The set of symbols that a programming tool uses for queries is entirely between the tool and the languages that choose to cooperate with it. For example, in addition to 'color-lexer, DrRacket uses a 'drracket:toolbar-buttons query to determine which buttons should be available in the toolbar to operate on modules using the language.

The syntax/module-reader language lets you specify get-info handling through a #:info optional specification. The protocol for an #:info function is slightly different from the raw get-info protocol; the revised protocol allows syntax/module-reader the possibility of handling future language-information queries automatically.

3.6. Module-Handling Configuration

Suppose that the file "death-list-5.rkt" contains

"death-list-5.rkt"

#lang racket         
(list "O-Ren Ishii"  
      "Vernita Green"
      "Budd"         
      "Elle Driver"  
      "Bill")        

If you require "death-list-5.rkt" directly, then it prints the list in the usual Racket result format:

> (require "death-list-5.rkt")                              
'("O-Ren Ishii" "Vernita Green" "Budd" "Elle Driver" "Bill")

However, if "death-list-5.rkt" is required by a "kiddo.rkt" that is implemented with scheme instead of racket:

"kiddo.rkt"

#lang scheme                
(require "death-list-5.rkt")

then, if you run "kiddo.rkt" file in DrRacket or if you run it directly with racket, "kiddo.rkt" causes "death-list-5.rkt" to print its list in traditional Scheme format, without the leading quote:

("O-Ren Ishii" "Vernita Green" "Budd" "Elle Driver" "Bill")

The "kiddo.rkt" example illustrates how the format for printing a result value can depend on the main module of a program instead of the language that is used to implement it.

More broadly, certain features of a language are only invoked when a module written in that language is run directly with racket (as opposed to being imported into another module). One example is result-printing style as shown above. Another example is REPL behavior. These features are part of whats called the run-time configuration of a language.

Unlike the syntax-coloring property of a language (as described in Source-Handling Configuration), the run-time configuration is a property of a module per se as opposed to a property of the source text representing the module. For that reason, the run-time configuration for a module needs to be available even if the module is compiled to bytecode form and the source is unavailable. Therefore, run-time configuration cannot be handled by the get-info function were exporting from the languages parser module.

Instead, it will be handled by a new configure-runtime submodule that well add inside the parsed module form. When a module is run directly with racket, racket looks for a configure-runtime submodule. If it exists, racket runs it. But if the module is imported into another module, the 'configure-runtime submodule is ignored. (And if the configure-runtime submodule doesnt exist, racket just evaluates the module as usual.) That means that the configure-runtime submodule can be used for any special setup tasks that need to happen when the module is run directly.

Going back to the literal language (see Source-Handling Configuration), we can adjust the language so that directly running a literal module causes it to print out its string, while using a literal module in a larger program simply provides data without printing. To make this work, we will need an extra module. (For clarity here, we will implement this module as a separate file. But it could equally well be a submodule of an existing file.)

.... (the main installation or the users space)   
|- "literal"                                       
   |- "main.rkt"            (with reader submodule)
   |- "show.rkt"            (new)                  
  • The "literal/show.rkt" module will provide a show function to be applied to the string content of a literal module, and also provide a show-enabled parameter that controls whether show actually prints the result.

  • The new configure-runtime submodule in "literal/main.rkt" will set the show-enabled parameter to #t. The net effect is that show will print the strings that its given, but only when a module using the literal language is run directly (because only then will the configure-runtime submodule be invoked).

These changes are implemented in the following revised "literal/main.rkt":

"literal/main.rkt"

#lang racket                                             
                                                         
(module reader racket                                    
  (require syntax/strip-context)                         
                                                         
  (provide (rename-out [literal-read read]               
                       [literal-read-syntax read-syntax])
           get-info)                                     
                                                         
  (define (literal-read in)                              
    (syntax->datum                                       
     (literal-read-syntax #f in)))                       
                                                         
  (define (literal-read-syntax src in)                   
    (with-syntax ([str (port->string in)])               
      (strip-context                                     
       #'(module anything racket                         
           (module configure-runtime racket              
             (require literal/show)                      
             (show-enabled #t))                          
           (require literal/show)                        
           (provide data)                                
           (define data 'str)                            
           (show data)))))                               
                                                         
  (define (get-info in mod line col pos)                 
    (lambda (key default)                                
      (case key                                          
        [(color-lexer)                                   
         (dynamic-require 'syntax-color/default-lexer    
                          'default-lexer)]               
        [else default]))))                               

Then the "literal/show.rkt" module must provide the show-enabled parameter and show function:

"literal/show.rkt"

#lang racket                             
                                         
(provide show show-enabled)              
                                         
(define show-enabled (make-parameter #f))
                                         
(define (show v)                         
  (when (show-enabled)                   
    (display v)))                        

With all of the pieces for literal in place, try running the following variant of "tuvalu.rkt" directly and through a require from another module:

"tuvalu.rkt"

#lang literal
Technology!  
System!      
Perfect!     

When run directly, well see the result printed like so, because our configure-runtime submodule will have set the show-enabled parameter to #t:

Technology! System! Perfect!

But when imported into another module, printing will be suppressed, because the configure-runtime submodule will not be invoked, and therefore the show-enabled parameter will remain at its default value of #f.