Repository navigation
fuzz with AFL++ - #87
yuval-block wants to merge 6 commits into
Conversation
1996cad to
8976732
Compare
range-diff for force push 1996cadd6e → 897673259c
|
AFL++ is a mutation engine with its own scheduling and its own take on
coverage, so it finds inputs the others do not. The targets wrap the
shared library, in a workspace of their own.
cd fuzz/afl && cargo afl build && cargo afl fuzz -i in -o out target/debug/afl_lattice_laws
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Package cargo-afl, which nixpkgs lacks. Build the AFL++ runtime and
afl-fuzz with it in one derivation, and the AFL++ targets against that
runtime in another, then add an app that runs a target with afl-fuzz
against a corpus in the checkout:
nix run .#fuzz-afl -- afl_lattice_laws -V 600
Only the corpus and afl-fuzz's output live outside the store, so a run
does not depend on whatever compiler and libraries the calling shell has.
cargo-afl builds the runtime from AFL++ sources bundled in
cargo-afl-common, which it finds by that crate's build-time
CARGO_MANIFEST_DIR, a path that is gone once cargo-afl is built. The
runtime derivation seeds cargo-afl's build directory with those sources
from the crate itself instead. An empty .git there makes cargo-afl skip
its own copy, so it does not need patching.
The check, in the nightly aggregate, builds the targets with crash_test's
panic compiled in and confirms AFL++ finds it within a minute, which it
only does with working coverage feedback. AFL_BENCH_UNTIL_CRASH stops
afl-fuzz at the first crash rather than waiting out the minute.
On macOS, afl-fuzz needs more System V shared memory than the defaults
allow, and raising them takes root, so AFL++ is built against POSIX
shared memory there.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Whether AFL++ solves crash_test within its time limit depends on its
random choices, so fuzz-afl-verify could pass or fail with no change to
its inputs. A check that flakes like that blames whichever commit happens
to be built, and a hard-to-find regression would point at the wrong one.
Run it as an app instead, so CI can run it on a schedule:
nix run .#fuzz-afl-verify
The time limit goes from 60s to 600s, since an app is not bounded by a
check's budget, and afl-fuzz arguments after -- override it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Once a day, confirm AFL++ still finds the planted crash_test panic, then fuzz the lattice laws for half an hour. afl-fuzz exits cleanly even when it finds crashes, so a separate step fails the run if any were saved, and uploads them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add an afl_scrub bin for the shared scrub target, and fuzz it on the schedule too. The workflow runs each target as its own matrix job, so a crash in one does not stop the other. With two targets, the app gives each its own output directory under fuzz/afl/out: afl-fuzz keeps its queue and crashes there, and a run of one target must not pick up another's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds a cargo-afl frontend over the shared targets and runs it through nix:
nix run .#fuzz-afl -- afl_lattice_laws -V 600
The AFL++ runtime, afl-fuzz and the targets are built in derivations and
the app only runs them against a corpus in the checkout.
Fuzzing is randomized, so it runs from a daily workflow rather than as a
flake check. That workflow first runs fuzz-afl-verify, which confirms
AFL++ finds the crash_test panic.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3e7c10e to
f23a148
Compare
8976732 to
b7a40f4
Compare
range-diff for force push 897673259c → b7a40f47a5
|
1 similar comment
range-diff for force push 897673259c → b7a40f47a5
|
| type = "app"; | ||
| meta.description = "Run an AFL++ target: nix run .#fuzz-afl -- <target> [afl-fuzz args]"; | ||
| program = lib.getExe ( | ||
| pkgs.writeShellApplication { |
There was a problem hiding this comment.
fuzz-afl app needs mkdir and ls from caller
writeShellApplication sets no runtimeInputs, so the wrapper's mkdir and ls (lines 116-117) resolve from the caller's PATH; with PATH=/nonexistent it exits 127 before starting afl-fuzz.
oak · automated · b7a40f4
| printf seed >"$scratch/in/seed" | ||
| # afl-fuzz's exit status says nothing useful here; what it left | ||
| # in its output directory does. | ||
| ${lib.getExe aflplusplus} -i "$scratch/in" -o "$scratch/out" -V 600 "$@" -- ${verifyTargets}/bin/afl_crash_test || true |
There was a problem hiding this comment.
doesn't find crash_test within this 600 s budget on x86_64 Linux at b7a40f4 (libFuzzer and honggfuzz take seconds), and 1 h of afl_scrub (82M execs) saves 0 crashes while the other two hit the #84 check within a minute
the 600 s miss is this host, not the search: core_pattern pipes to apport here, so a crash takes ~1.1 s and AFL++'s 20 ms timeout files it as a timeout (verify saved 5), and AFL_I_DONT_CARE_ABOUT_MISSING_CRASHES hides AFL++'s warning about that. with -t 3000 verify finds crash_test in 10 s. afl_scrub still saves 0 crashes and 0 timeouts with -t 3000 (2 min), so there it does look like the search
afl_lattice_laws: 162M execs, nothing
AFL++ commands
nix run .#fuzz-afl-verify
nix run .#fuzz-afl-verify -- -t 3000
nix run .#fuzz-afl -- afl_scrub -V 3600
nix run .#fuzz-afl -- afl_lattice_laws -V 3600
| # afl-fuzz's exit status says nothing useful here; what it left | ||
| # in its output directory does. | ||
| ${lib.getExe aflplusplus} -i "$scratch/in" -o "$scratch/out" -V 600 "$@" -- ${verifyTargets}/bin/afl_crash_test || true | ||
| crashes=("$scratch"/out/default/crashes/id:*) |
There was a problem hiding this comment.
Distributed AFL args cause false 'no crash'
The app advertises [afl-fuzz args], but -S/-M make AFL++ save crashes under out/<instance>/crashes while the script checks only out/default; it then prints AFL++ did not find the crash and exits non-zero despite a saved crash.
oak · automated · b7a40f4
| inherit pnameSuffix RUSTFLAGS; | ||
| nativeBuildInputs = [ cargo-afl ]; | ||
| XDG_DATA_HOME = "${aflplusplus}/share"; | ||
| buildPhaseCargoCommand = "cargo afl build --release --bins"; |
There was a problem hiding this comment.
Fuzz targets are built for the host CPU
cargo afl build appends -C target-cpu=native, so the fuzz targets are compiled for the builder's CPU and the derivation output depends on the machine that built it.
oak · automated · b7a40f4
Adds a cargo-afl frontend over the shared targets and runs it through nix:
The AFL++ runtime, afl-fuzz and the targets are built in derivations and
the app only runs them against a corpus in the checkout.
There is a binary per target, afl_lattice_laws and afl_scrub, and the
workflow fuzzes each in a job of its own.
Fuzzing is randomized, so it runs from a daily workflow rather than as a
flake check. That workflow first runs fuzz-afl-verify, which confirms
AFL++ finds the crash_test panic.
🤖 Generated with Claude Code