Architecture¶
The crate is a library (src/lib.rs) plus a thin command-line front end (src/main.rs). This
page is an overview; the doc comments in each module go into detail.
Compiler passes¶
Lower the checked AST to IR: blocks of instructions over virtual registers, one per local.
Inline small functions that call no others, then optimize each function: hoist and reuse index and uniqueness checks, rotate loops, and drop dead instructions.
For each function, compute live intervals and assign registers by linear scan, spilling the least-used values to the stack.
Emit x86-64 or arm64 assembly from the allocated IR, then assemble it with the built-in assembler and link it with the built-in linker into a static ELF executable for Linux, a signed Mach-O executable for macOS, or a PE executable for Windows.
Pipelines¶
src/driver.rs wires the stages together, and src/main.rs picks a pipeline from the command line.
stone run:project(lexer->parserfor each file, then link) ->checker->interpreterstone build:project->checker->codegen::ir(lower to IR, thenir::inlineandir::opt) ->codegen::regalloc(linear scan) ->codegen::x64orcodegen::arm64(emit), picked by--target, which defaults to the host, ->codegen::asm(assemble) ->codegen::elf,codegen::macho, orcodegen::pe(link), by the target’sOsstone check:project->checker->unused, printing every diagnostic and unused-name warning without runningstone fmt:format, a token formatter that keeps commentsstone add,remove,fetch, andtree:deps, overpackage
project::link lexes and parses the entry file, loads every module its use statements name, and
merges them into one module in which each library function is renamed to its module path, such
as geometry.shapes.area, and every use of it points there. It also parses src/prelude.stn,
which defines option, result, and the prelude’s traits, into every program, adds the source
of net or gui when a program uses them, and turns each lambda into a function of its own
(lift). Everything after it sees a program
of one file, except that each Span carries the FileId of the file it is in.
The functions of a struct’s impl blocks are named by the struct’s linked name and their own,
as geometry.Point.shift. Which one a call such as p.shift(1.0) calls depends on the type of
p, so the checker resolves every such call and records it, by the call’s span, in
Analysis::methods, which both the interpreter and lowering follow.
Source map¶
module |
role |
|---|---|
|
source positions ( |
|
errors and warnings with a span, and their terminal rendering |
|
|
|
source text to tokens, including Python-style |
|
syntax tree nodes, modeled on |
|
recursive-descent PEG parser; |
|
type inference and name resolution, producing diagnostics, expression types, and symbols with their references |
|
turns lambdas into functions, and those that use outside variables into closures holding copies |
|
where each variable is used for the last time, so both backends move values there |
|
warnings about names a program defines and never uses, for |
|
tree-walking evaluator, over a tree made of each body once with variables resolved to slots |
|
|
|
the |
|
|
|
GPU kernels for arithmetic on lists: the device IR ( |
|
the built-in assembler: parses the GNU as syntax both backends emit and encodes it ( |
|
the built-in linker for Linux: lays out sections, resolves fixups, and writes a static ELF executable with a symbol table |
|
the built-in linker for macOS: a Mach-O executable that loads only |
|
the built-in linker for Windows: a PE32+ executable that imports only from the system’s DLLs |
|
what both backends share about the program being compiled: the checked and lowered program, labels, interned strings, runtime failures, and which runtime routines it needs |
|
the IR the backend compiles through: lowering from the AST ( |
|
target-independent linear-scan register allocation and parallel-move ordering |
|
the x86-64 backend: instruction selection from allocated IR ( |
|
the arm64 backend, with the same split ( |
|
the builtins shared by both backends ( |
|
loading the modules a program uses ( |
|
packages: reading |
|
the |
|
|
|
the run/build/check pipelines, and |
Language server¶
lsp/ is the stone-lsp crate, a language server built on lsp-server and lsp-types, kept out of the main crate so stone itself still depends only on clap. On every change it reanalyzes each open document as part of its program (the one whose entry file is the nearest main.stn above it, or the lib.stn beside the nearest stone.toml) with project::link and driver::analyze_linked, reading open documents’ text over what is on disk, and answers requests from the resulting checker::Analysis: its diagnostics, the type of every expression, and every symbol with all of its references, in every file.
request |
answered from |
|---|---|
diagnostics |
|
hover |
the symbol’s signature, a builtin’s documentation, or the innermost expression’s type |
definition, references, rename |
|
document symbols |
globals and functions, with each function’s parameters and locals |
completion |
|
formatting |
|
editors/vscode/ is a VS Code extension that provides highlighting and indentation rules and starts stone-lsp for .stn files.
Fuzzing¶
The fuzz/ crate uses cargo-fuzz (libFuzzer, nightly Rust). It is a separate crate, so the main crate still depends only on clap.
target |
input |
checks |
|---|---|---|
|
arbitrary text |
the front end returns tokens, a module, or an error, and never panics, overflows the stack, or takes exponential time |
|
arbitrary text |
formatting returns an error or text that formatting again leaves unchanged |
|
arbitrary text |
the interpreter finishes under |
|
arbitrary text that parses |
both backends’ |
|
programs from |
the same stages on deep, valid programs, for both backends |
|
programs from |
|
stone_fuzz::generate writes source text rule by rule from the grammar, tracking scope so every name and call is defined and every program passes the checker. It covers int and float arithmetic, comparisons, functions with up to eight parameters, if/while/for with break and cont, multi-argument print, strings, lists, and, through fuzz/src/compound.rs, structs with impl blocks, enums, options, maps, and lists of them. Every loop is bounded, and functions never read globals, which may not be assigned yet when they run. differential skips a program if the interpreter rejects it (out of fuel, division by zero, an unassigned variable).
Seeds for the text targets are the .stn programs in fuzz/corpus/<target>/. fuzz/stone.dict lists stone’s tokens.
Benchmarks¶
The bench/ crate compares compiled stone with C (gcc -O2 and gcc -O0) and Python. It is a separate crate, like fuzz/, and depends on stone only for its tests. Each benchmark in bench/programs/ is written three times, as .stn, .c, and .py, which all read the workload size from the arguments in a .args file, and the runner refuses to time a program unless all four builds print the same .out file. See bench/README.md for the programs and a baseline.