Fuzzing¶
The fuzz/ crate uses cargo-fuzz (libFuzzer), which
needs nightly Rust:
rustup toolchain install nightly
cargo install cargo-fuzz
cargo +nightly fuzz run parse -- -dict=fuzz/stone.dict
Targets¶
target |
input |
checks |
|---|---|---|
|
arbitrary text |
the front end returns tokens, a module, or an error, and never panics, overflows the stack, or takes exponential time |
|
arbitrary text |
formatting returns an error or text that formatting again leaves unchanged, and never panics |
|
arbitrary text |
the interpreter finishes under |
|
arbitrary text that parses |
both backends’ |
|
generated programs |
the same stages on deep, valid programs, which both backends must compile, assemble, and link |
|
generated programs |
|
stone_fuzz::generate writes valid programs rule by rule from the grammar, tracking scope so
every name and call is defined and every program passes the checker. Besides ints, floats, and
bools, they use strings, structs with impl blocks, enums, options, maps, and lists of them,
under rules in fuzz/src/compound.rs that keep them from failing at runtime. Seeds for the text targets
are the .stn programs in fuzz/corpus/<target>/, and fuzz/stone.dict lists stone’s tokens.
cargo test --manifest-path fuzz/Cargo.toml checks the generator on stable Rust, including
that generated programs pass the checker and format stably, that 2,000 of them assemble and link
for every target, and that 150 of them print the same under both backends and free everything
they allocate, plus the same checks on longer programs, which reach compound values more often. The compiled side builds
for the host’s processor unless STONE_FUZZ_TARGET names the other one, such as
STONE_FUZZ_TARGET=aarch64 on an x86-64 machine, which runs the program under qemu-user, as
the golden tests do.
This works for the differential target too.
When a fuzzer finds a crash¶
The failing input is saved under fuzz/artifacts/<target>/. Reproduce it with
cargo +nightly fuzz run parse fuzz/artifacts/parse/crash-<hash>
then fix the bug and turn the input into a regression test: a unit test, or a program in
tests/programs/ with its .out (and .err if it should fail at runtime).