14.14 Linklets and the Core Compiler

► |  | The Racket Reference

1 | Language Model
2 | Notation for Documentation
3 | Syntactic Forms
4 | Datatypes
5 | Structures
6 | Classes and Objects
7 | Units
8 | Contracts
9 | Pattern Matching
10 | Control Flow
11 | Concurrency and Parallelism
12 | Macros
13 | Input and Output
14 | Reflection and Security
15 | Operating System
16 | Memory Management
17 | Unsafe Operations
18 | Running Racket
| Bibliography
| Index

▼ | 14 | Reflection and Security

14.1 | Namespaces
14.2 | Evaluation and Compilation
14.3 | The racket/ load Language
14.4 | Module Names and Loading
14.5 | Impersonators and Chaperones
14.6 | Security Guards
14.7 | Custodians
14.8 | Thread Groups
14.9 | Structure Inspectors
14.10 | Code Inspectors
14.11 | Plumbers
14.12 | Sandboxed Evaluation
14.13 | The racket/ repl Library
14.14 | Linklets and the Core Compiler
14.15 | Deprecation

On this page:

linklet?
compile-linklet
recompile-linklet
eval-linklet
instantiate-linklet
linklet-import-variables
linklet-export-variables
linklet-add-target-machine-info
linklet-summarize-target-machine-info
decompile-linklet
linklet-directory?
hash->linklet-directory
linklet-directory->hash
linklet-bundle?
hash->linklet-bundle
linklet-bundle->hash
linklet-body-reserved-symbol?
instance?
make-instance
instance-name
instance-data
instance-variable-names
instance-variable-value
instance-set-variable-value!
instance-unset-variable!
instance-describe-variable!
variable-reference->instance
correlated?
correlated-source
correlated-line
correlated-column
correlated-position
correlated-span
correlated-e
correlated->datum
datum->correlated
correlated-property
correlated-property-symbol-keys

Racket
top contents ← prev up next →

### 14.14 Linklets and the Core Compiler🔗ℹ

(require racket/linklet) | package: base

A linklet is a primitive element of compilation, bytecode marshaling, and evaluation. Racket’s implementations of modules, macros, and top-level evaluation are all built on linklets. Racket programmers generally do not encounter linklets directly, but the racket/linklet library provides access to linklet facilities.

A single Racket module (or collection of top-level forms) is typically implemented by multiple linklets. For example, each phase of evaluation that exists in a module is implemented in a separate linklet. A linklet is also used for metadata such as the module path indexes for a module’s requires. These linklets, plus some other metadata, are combined to form a linklet bundle. Information in a linklet bundle is keyed by either a symbol or a fixnum. A linklet bundle containing linklets can be marshaled to and from a byte stream by write and (with read-accept-compiled is enabled) read. A compiled form in the sense of compiled-expression? (such as the result from compile) may be a linklet bundle.

When a Racket module has submodules, the linklet bundles for the module and the submodules are grouped together in a linklet directory. A linklet directory can have nested linklet directories. Information in a linklet directory is keyed by #f or a symbol, where #f must be mapped to a linklet bundle (if anything) and each symbol must be mapped to a linklet directory. A linklet directory can be equivalently viewed as a mapping from a lists of symbols to a linklet bundle. Like linklet bundles, a linklet directory can be marshaled to and from a byte stream by write and read; the marshaled form allows individual linklet bundles to be loaded independently. A compiled form in the sense of compiled-expression? (such as the result from compile) may be a linklet directory.

A linklet consists of a set of variable definitions and expressions, an exported subset of the defined variable names, a set of variables to export from the linklet despite having no corresponding definition, and a set of imports that provide other variables for the linklet to use. To run a linklet, it is instantiated as as linklet instance (or just instance, for short). When a linklet is instantiated, it receives other linklet instances for its imports, and it extracts a specified set of variables that are exported from each of the given instances. The newly created linklet instance provides its exported variables for use by other linklets or for direct access via instance-variable-value. A linklet instance can be synthesized directly with make-instance.

A linklet is created by compiling an enriched S-expression representation of its source. Since linklets exist below the layer of macros and syntax objects, linklet compilation does not use syntax objects. Instead, linklet compilation uses correlated objects, which are like syntax objects without lexical-context information and without the constraint that content is coerced to correlated objects. Using an S-expression or correlated object, the grammar of a linklet as recognized by compile-linklet is

> > (linklet [[imported-id/renamed ...] ...]
> > [exported-id/renamed ...]
> > defn-or-expr ...)
> > imported-id/renamed |  | = |  | imported-id
> > |  | | |  | (external-imported-id internal-imported-id)
> > exported-id/renamed |  | = |  | exported-id
> > |  | | |  | (internal-exported-id external-exported-id)

Each import set [imported-id/renamed ...] refers to a single imported instance, and each import-id/renamed corresponds to a variable from that instance. If separate external-imported-id and internal-imported-id are specified, then external-imported-id is the name of the variable as exported by the instance, and internal-imported-id is the name used to refer to the variable in the defn-or-exprs. For exports, separate internal-exported-id and external-exported-id names corresponds to the variable name as exported as referenced in the defn-or-exprs, respectively.

The grammar of an defn-or-expr is similar to the expander’s grammar of fully expanded expressions (see Fully Expanded Programs) with some exceptions: quote-syntax and #%top are not allowed; #%plain-lambda is spelled lambda; #%plain-app is omitted (i.e., application is implicit); lambda, case-lambda, let-values, and letrec-values can have only a single body expression; begin-unsafe is like begin in an expression position, but its body is compiled in unsafe mode; and numbers, booleans, strings, and byte strings are self-quoting. Primitives are accessed directly by name, and shadowing is not allowed within a linklet form for primitive names (see linklet-body-reserved-symbol?), imported variables, defined variables, or local variables.

When an exported-id/renamed has no corresponding definition among the defn-or-exprs, then the variable is effectively defined as uninitialized; referencing the variable will trigger exn:fail:contract:variable, the same as referencing a variable before it is defined. When a target instance is provided to instantiate-linklet, any existing variable with the same name will be left as-is, instead of set to undefined. This treatment of uninitialized variables provides core support for top-level evaluation where variables may be referenced and then defined in a separate element of compilation.

Added in version 6.90.0.1 of package base.

> procedure (linklet? v) → boolean?
> v : any/c

Returns #t if v is a linklet, #f otherwise.

> > procedure
> > (compile-linklet |  | form |  |  |  |
> > | [ | info |  |  |  |
> > |  | import-keys |  |  |  |
> > |  | get-import |  |  |  |
> > |  | options]) |  | → |  | linklet?
> form : (or/c correlated? any/c)
> info : (or/c hash? any/c) = #f
> import-keys : #f = #f
> get-import : #f = #f
> | options |  | : |  |
> (listof (or/c 'serializable 'unsafe 'static 'quick
> 'use-prompt 'unlimited-compile
> 'uninterned-literal))
> |  |  | = |  | '(serializable)
> (compile-linklet |  | form |  |  |  |
> |  | info |  |  |  |
> |  | import-keys |  |  |  |
> | [ | get-import |  |  |  |
> |  | options]) |  | → |  |
> linklet? |  | vector?
> form : (or/c correlated? any/c)
> info : (or/c hash? any/c)
> import-keys : vector?
> | get-import |  | : |  |
> (or/c #f (any/c . -> . (values (or/c linklet? instance? #f)
> (or/c vector? #f))))
> |  |  | = |  | #f
> | options |  | : |  |
> (listof (or/c 'serializable 'unsafe 'static 'quick
> 'use-prompt 'unlimited-compile
> 'uninterned-literal))
> |  |  | = |  | '(serializable)

Takes an S-expression or correlated object for a linklet form and produces a linklet. As long as 'serializable included in options, the resulting linklet can be marshaled to and from a byte stream when it is part of a linklet bundle (possibly in a linklet directory).

The optional info hash provides various debugging details about the linklet, such as the module name the linklet is part of, the linklet name, and the phase for body linklets. If a 'name value is present in the hash, it is associated to the linklet for debugging purposes and as the default name of the linklet’s instance. If info is not a hash, it is assumed to be a name value directly for backward compatibility.

The optional import-keys and get-import arguments support cross-linklet optimization. If import-keys is a vector, it must have as many elements as sets of imports in form. If the compiler becomes interested in optimizing a reference to an imported variable, it passes back to get-import (if non-#f) the element of import-keys that corresponds to the variable’s import set. The get-import function can then return a linklet or instance that represents an instance to be provided to the compiled linklet when it is eventually instantiated; ensuring consistency between reported linklet or instance and the eventual instance is up to the caller of compile-linklet, but see also linklet-add-target-machine-info. If get-import returns #f as its first value, the compiler will be prevented from making any assumptions about the imported instance. The second result from get-import is an optional vector of keys to provide transitive information on a returned linklet’s imports (and is not allowed for a returned instance); the returned vector must have the same number of elements as the linklet has imports. When vector elements are eq? and non-#f, the compiler can assume that they correspond to the same run-time instance. A #f value for get-import is equivalent to a function that always returns two #f results.

When import-keys is not #f, then the compiler is allowed to grow or shrink the set of imported instances for the linklet. The result vector specifies the keys of the imports for the returned linklet. Any key that is #f or a linklet instance must be preserved intact, however.

If 'unsafe is included in options, then the linklet is compiled in unsafe mode: uses of safe operations within the linklet can be converted to unsafe operations on the assumption that the relevant contracts are satisfied. For example, car is converted to unsafe-car. Some substituted unsafe operations may not have directly accessible names, such as the unsafe variant of in-list that can be substituted in unsafe mode. An unsafe operation is substituted only if its (unchecked) contract is subsumed by the safe operation’s contract. The fact that the linklet is compiled in unsafe mode can be exposed through variable-reference-from-unsafe? using a variable reference produced by a #%variable-reference form within the module body. Within a linklet an individual expression can be compiled in unsafe mode by wrapping it in begin-unsafe; when a whole linklet is compiled in unsafe mode, begin-unsafe is redundant and ignored.

If 'static is included in options, then the linklet must be instantiated only once; if the linklet is serialized, then any individual instance read from the serialized form must be instantiated at most once. Compilation with 'static is intended to improve the performance of references within the linklet to defined and imported variables.

If 'quick is included in options, then linklet compilation may trade run-time performance for compile-time performance—that is, spend less time compiling the linklet, but the resulting linklet may run more slowly.

If 'use-prompt is included in options, then instantiating resulting linklet always wraps a prompt around each definition and immediate expression in the linklet. Otherwise, supplying #t as the use-prompt? argument to instantiate-linklet may only wrap a prompt around the entire instantiation.

If 'unlimited-compile is included in options, then compilation never falls back to interpreted mode for an especially large linklet. See also CS Compilation Modes.

If 'uninterned-literal is included in options, then literals in form will not necessarily be interned via datum-intern-literal when compiling or loading the linklet. Disabling the use of datum-intern-literal can be especially useful of the linklet includes a large string or byte string constant that is not meant to be shared.

The symbols in options must be distinct, otherwise exn:fail:contract exception is raised.

Changed in version 7.1.0.8 of package base: Added the 'use-prompt option.
Changed in version 7.1.0.10: Added the 'uninterned-literal option.
Changed in version 7.5.0.14: Added the 'quick option.
Changed in version 8.11.1.2: Changed info to a hash.
Changed in version 8.13.0.9: Added the 'unlimited-compile option.

> > procedure
> > (recompile-linklet |  | linklet |  |  |  |
> > | [ | info |  |  |  |
> > |  | import-keys |  |  |  |
> > |  | get-import |  |  |  |
> > |  | options]) |  | → |  | linklet?
> linklet : linklet?
> info : (or/c hash? any/c) = #f
> import-keys : #f = #f
> get-import : #f = #f
> | options |  | : |  |
> (listof (or/c 'serializable 'unsafe 'static 'quick
> 'use-prompt 'uninterned-literal))
> |  |  | = |  | '(serializable)
> (recompile-linklet |  | linklet |  |  |  |
> |  | info |  |  |  |
> |  | import-keys |  |  |  |
> | [ | get-import |  |  |  |
> |  | options]) |  | → |  |
> linklet? |  | vector?
> linklet : linklet?
> info : (or/c hash? any/c)
> import-keys : vector?
> | get-import |  | : |  |
> (or/c (any/c . -> . (values (or/c linklet? #f)
> (or/c vector? #f)))
> #f)
> |  |  | = |  | (lambda (import-key) (values #f #f))
> | options |  | : |  |
> (listof (or/c 'serializable 'unsafe 'static 'quick
> 'use-prompt 'uninterned-literal))
> |  |  | = |  | '(serializable)

Like compile-linklet, but takes an already-compiled linklet and potentially optimizes it further.

Changed in version 7.1.0.6 of package base: Added the options argument.
Changed in version 7.1.0.8: Added the 'use-prompt option.
Changed in version 7.1.0.10: Added the 'uninterned-literal option.
Changed in version 7.5.0.14: Added the 'quick option.
Changed in version 8.11.1.2: Changed info to a hash.

> procedure (eval-linklet linklet) → linklet?
> linklet : linklet?

Returns a variant of a linklet that is prepared for JIT compilation such that every later use of the result linklet with instantiate-linklet shares the JIT-generated code. However, the result of eval-linklet cannot be marshaled to a byte stream as part of a linklet bundle, and it cannot be used with recompile-linklet.

> > procedure
> > (instantiate-linklet |  | linklet |  |  |  |
> > |  | import-instances |  |  |  |
> > | [ | target-instance? |  |  |  |
> > |  | use-prompt?]) |  | → |  | instance?
> linklet : linklet?
> import-instances : (listof instance?)
> target-instance? : #f = #f
> use-prompt? : any/c = #t
> (instantiate-linklet |  | linklet |  |  |  |
> |  | import-instances |  |  |  |
> |  | target-instance |  |  |  |
> | [ | use-prompt?]) |  | → |  | any
> linklet : linklet?
> import-instances : (listof instance?)
> target-instance : instance?
> use-prompt? : any/c = #t

Instantiates linklet by running its definitions and expressions, using the given import-instances for its imports. The number of instances in import-instances must match the number of import sets in linklet.

If target-instance is #f or not provided, the result is a fresh instance for the linklet. If target-instance is an instance, then the instance is used and modified for the linklet definitions and expressions, and the result is the value of the last expression in the linklet.

The linklet’s exported variables are accessible in the result instance or in target-instance using the linklet’s external name for each export. If target-instance is provided as non-#f, its existing variables remain intact if they are not modified by a linklet definition.

If use-prompt? is true, then a a prompt is wrapped around the linklet instantiation in same ways as an expression in a module body. If the linklet contains multiple definitions or immediate expressions, then a prompt may or may not be wrapped around each definition or expression; supply 'use-prompt to compile-linklet to ensure that a prompt is used around each definition and expression.

> procedure (linklet-import-variables linklet) → (listof (listof symbol?))
> linklet : linklet?

Returns a description of a linklet’s imports. Each element of the result list corresponds to an import set as satisfied by a single instance on instantiation, and each member of the set is a variable name that is used from the corresponding imported instance.

> procedure (linklet-export-variables linklet) → (listof symbol?)
> linklet : linklet?

Returns a description of a linklet’s exports. Each element of the list corresponds to a variable that is made available by the linklet in its instance.

> > procedure
> > (linklet-add-target-machine-info |  | linklet |  |  |  |
> > |  | from-linklet) |  | → |  | linklet?
> linklet : linklet?
> from-linklet : (or linklet? hash?)

When compile-linklet or recompile-linklet requests a linklet via get-import for cross-module information, the linklet is expected to have information compatible with the current compilation target as determined by current-compile-target-machine. To simplify the management of linklets to both run and use for cross-compilation, a linklet implementation may support information for multiple target machines within a linklet, in which case linklet-add-target-machine-info returns a linklet like linklet but with target-specific information added from from-linklet. The two linklets must be from compatible sources, but linklet-add-target-machine-info might perform only a sanity check for compatibility.

The from-linklet can be a linklet or a summary of a linklet’s information as produced by linklet-summarize-target-machine-info.

Added in version 8.12.0.3 of package base.
Changed in version 8.17.0.3: Added support for from-linklet as a summary.

> procedure (linklet-summarize-target-machine-info from-linklet) → hash?
> from-linklet : linklet?

Returns a value that has the same information as from-linklet for linklet-add-target-machine-info, but in a form that can be portably serialized via racket/fasl.

Added in version 8.17.0.3 of package base.

> procedure (decompile-linklet linklet) → (or/c #f correlated? any/c)
> linklet : linklet?

Attempts to recompile a linklet back into the S-expression form that compile-linklet expects. If the linklet cannot be decompiled, the result is #f. A linklet that is generated via compile with current-compile-target-machine set to #f (for machine-independent bytecode) always can be decompiled.

Added in version 8.18.0.19 of package base.

> procedure (linklet-directory? v) → boolean?
> v : any/c

Returns #t if v is a linklet directory, #f otherwise.

> procedure (hash->linklet-directory content) → linklet-directory?
> content : (and/c hash? hash-eq? immutable? (not/c impersonator?))

Constructs a linklet directory given mappings in the form of a hash table. Each key of content must be either a symbol or #f, each symbol must be mapped to a linklet directory, and #f must be mapped to a linklet bundle or not mapped.

> procedure (linklet-directory->hash linklet-directory)
> → (and/c hash? hash-eq? immutable? (not/c impersonator?))
> linklet-directory : linklet-directory?

Extracts the content of a linklet directory into a hash table.

> procedure (linklet-bundle? v) → boolean?
> v : any/c

Returns #t if v is a linklet bundle, #f otherwise.

> procedure (hash->linklet-bundle content) → linklet-bundle?
> content : (and/c hash? hash-eq? immutable? (not/c impersonator?))

Constructs a linklet bundle given mappings in the form of a hash table. Each key of content must be either a symbol or a fixnum. Values in the hash table are unconstrained, but the intent is that they are all linklets or values that can be recovered from write output by read.

> procedure (linklet-bundle->hash linklet-bundle)
> → (and/c hash? hash-eq? immutable? (not/c impersonator?))
> linklet-bundle : linklet-bundle?

Extracts the content of a linklet bundle into a hash table.

> procedure (linklet-body-reserved-symbol? sym) → boolean?
> sym : symbol?

Return #t if sym is a primitive name or other identifier that is not allowed as a binding within a linklet, #f otherwise.

Added in version 8.2.0.1 of package base.

> procedure (instance? v) → boolean?
> v : any/c

Returns #t if v is a linklet instance, #f otherwise.

> > procedure
> > (make-instance |  | name |  |  |  |
> > | [ | data |  |  |  |
> > |  | mode] |  |  |  |
> > |  | variable-name |  |  |  |
> > |  | variable-value ... |  |  |  |
> > |  | ...) |  | → |  | instance?
> name : any/c
> data : any/c = #f
> mode : (or/c #f 'constant 'consistent) = #f
> variable-name : symbol?
> variable-value : any/c

Constructs a linklet instance directly. Besides associating an arbitrary name and data value to the instance, the instance is populated with variables as specified by variable-name and variable-value.

The optional data and mode arguments must be provided if any variable-name and variable-value arguments are provided. The mode argument is used as in instance-set-variable-value! for every variable-name.

> procedure (instance-name instance) → any/c
> instance : instance?

Returns the value associated to instance as its name—either the first value provided to make-instance or the name of a linklet that was instantiated to create the instance.

> procedure (instance-data instance) → any/c
> instance : instance?

Returns the value associated to instance as its data—either the second value provided to make-instance or the default #f.

> procedure (instance-variable-names instance) → (list symbol?)
> instance : instance?

Returns a list of all names for all variables accessible from instance.

> > procedure
> > (instance-variable-value |  | instance |  |  |  |
> > |  | name |  |  |  |
> > | [ | fail-k]) |  | → |  | any
> instance : instance?
> name : symbol?
> fail-k : any/c = (lambda () (error ....))

Returns the value of the variable exported as name from instance. If no such variable is exported, then fail-k is used in the same way as by hash-ref.

> > procedure
> > (instance-set-variable-value! |  | instance |  |  |  |
> > |  | name |  |  |  |
> > |  | v |  |  |  |
> > | [ | mode]) |  | → |  | void?
> instance : instance?
> name : symbol?
> v : any/c
> mode : (or/c #f 'constant 'consistent) = #f

Sets or creates the variable exported as name in instance so that its value is v, as long as the variable does not exist already as constant. If a variable for name exists as constant, the exn:fail:contract exception is raised.

If mode is 'constant or 'consistent, then the variable is created or changed to be constant. Furthermore, when the instance is reported for a linklet’s import though a get-import callback to compile-linklet, the compiler can assume that the variable will be constant in all future instances that are used to satisfy a linklet’s imports.

If mode is 'consistent, when the instance is reported though a callback to compile-linklet, the compiler can further assume that the variable’s value will be the same for future instances. For compilation purposes, “the same” can mean that a procedure value will have the same arity and implementation details, a structure type value will have the same configuration, a marshalable constant will be equal? to the current value, and so on.

> procedure (instance-unset-variable! instance name) → void?
> instance : instance?
> name : symbol?

Changes instance so that it does not export a variable as name, as long as name does not exist as a constant variable. If a variable for name exists as constant, the exn:fail:contract exception is raised.

> > procedure
> > (instance-describe-variable! |  | instance |  |  |  |
> > |  | name |  |  |  |
> > |  | desc-v) |  | → |  | void?
> instance : instance?
> name : symbol?
> desc-v : any/c

Registers information about name in instance that may be useful for compiling linklets where the instance is returned via the get-import callback to compile-linklet. The desc-v description can be any value; the recognized descriptions depend on virtual machine, but may include the following:

- `(procedure ,arity-mask) — the value is always a procedure that is not impersonated and not a structure, and its arity in the style of procedure-arity-mask is arity-mask.

- `(procedure/succeeds ,arity-mask) — like `(procedure ,arity-mask), but for a procedure that never raises an exception of otherwise captures or escapes the calling context.

- `(procedure/pure ,arity-mask) — like `(procedure/succeeds ,arity-mask), but with no observable side effects, so a call to the procedure can be reordered.

Added in version 7.1.0.8 of package base.

> > procedure
> > (variable-reference->instance |  | varref |
> > | [ | ref-site?]) |
> → (if ref-site? (or/c instance? #f symbol?) instance?)
> varref : variable-reference?
> ref-site? : any/c = #f

Extracts the instance where the variable of varref is defined if ref-site? is #f, and returns the instance where varref itself resides if ref-site? is true. This notion of variable reference is the same as at the module level and can reflect the linklet instance that implements a particular phase of a module instance.

When ref-site? is #f, the result is #f when varref is from (#%variable-reference) with no identifier. The result is a symbol if varref refers to a primitive.

> procedure (correlated? v) → boolean?
> v : any/c
> procedure (correlated-source crlt) → any
> crlt : correlated?
> procedure (correlated-line crlt) → (or/c exact-positive-integer? #f)
> crlt : correlated?
> procedure (correlated-column crlt) → (or/c exact-nonnegative-integer? #f)
> crlt : correlated?
> procedure (correlated-position crlt) → (or/c exact-positive-integer? #f)
> crlt : correlated?
> procedure (correlated-span crlt) → (or/c exact-nonnegative-integer? #f)
> crlt : correlated?
> procedure (correlated-e crlt) → any
> crlt : correlated?
> procedure (correlated->datum crlt) → any
> crlt : (or/c correlated? any/c)
> procedure (datum->correlated v [srcloc prop]) → correlated?
> v : any/c
> | srcloc |  | : |  |
> (or/c correlated? #f
> (list/c any/c
> (or/c exact-positive-integer? #f)
> (or/c exact-nonnegative-integer? #f)
> (or/c exact-positive-integer? #f)
> (or/c exact-nonnegative-integer? #f))
> (vector/c any/c
> (or/c exact-positive-integer? #f)
> (or/c exact-nonnegative-integer? #f)
> (or/c exact-positive-integer? #f)
> (or/c exact-nonnegative-integer? #f)))
> |  |  | = |  | #f
> prop : (or/c correlated? #f) = #f
> procedure (correlated-property crlt key val) → correlated?
> crlt : correlated?
> key : any/c
> val : any/c
> (correlated-property crlt key) → any/c
> crlt : correlated?
> key : any/c
> procedure (correlated-property-symbol-keys crlt) → list?
> crlt : correlated?

Like syntax?, syntax-source, syntax-line, syntax-column, syntax-position, syntax-span, syntax-e, syntax->datum, datum->syntax, syntax-property, and syntax-property-symbol-keys, but for correlated objects.

Unlike datum->syntax, datum->correlated does not recur through the given S-expression and convert pieces to correlated objects. Instead, a correlated object is simply wrapped around the immediate value. In contrast, correlated->datum recurs through its argument (which is not necessarily a correlated object) to discover any correlated objects and convert them to plain S-expressions.

Changed in version 7.6.0.6 of package base: Added the prop argument to datum->correlated.

top contents ← prev up next →