Repository navigation
fuzz with libFuzzer - #85
yuval-block wants to merge 6 commits into
Conversation
| }: | ||
| let | ||
| # Every target in fuzz/libfuzzer, instrumented by cargo-fuzz and built | ||
| # in the sandbox, so that the app and the check run the same binaries. |
There was a problem hiding this comment.
Check runs different binaries than the app
The comment says the app and the check run the same binaries, but the check builds verifyTargets with RUSTFLAGS = "--cfg fuzzer_verify_crash" while the app uses targets without it; the app's fuzz_crash_test exits cleanly on the verification input while the check's panics, so the check's crash cannot be reproduced with nix run .#fuzz-libfuzzer.
RUSTFLAGS = "--cfg fuzzer_verify_crash";
— fungi-protocol/concurrent-psbt@f2419ee · nix/fuzz/libfuzzer.nix#L69
oak · automated · 1a059c9
| # The same layout cargo fuzz run uses, relative to the checkout. | ||
| corpus="fuzz/libfuzzer/corpus/$target" | ||
| artifacts="fuzz/libfuzzer/artifacts/$target" | ||
| mkdir -p "$corpus" "$artifacts" |
There was a problem hiding this comment.
Fuzzer app wrapper depends on caller's mkdir
The wrapper calls mkdir but declares no runtimeInputs, so mkdir is resolved from the caller's PATH; with a PATH that lacks mkdir, nix run .#fuzz-libfuzzer fails with mkdir: command not found before the fuzzer starts. Add pkgs.coreutils to runtimeInputs.
oak · automated · 1a059c9
| text = '' | ||
| target="''${1:?usage: nix run .#fuzz-libfuzzer -- <target> [libFuzzer args...]}" | ||
| shift | ||
| # The same layout cargo fuzz run uses, relative to the checkout. |
There was a problem hiding this comment.
Corpus paths are relative to CWD, not checkout
The script creates fuzz/libfuzzer/{corpus,artifacts} relative to the current directory, so running nix run /path/to/repo#fuzz-libfuzzer from a directory with no corpus silently starts a fresh corpus there; the comment says the layout is relative to the checkout.
oak · automated · 1a059c9
1a059c9 to
9f7f89b
Compare
range-diff for force push 1a059c9103 → 9f7f89b182
|
cargo-fuzz is the frontend most Rust fuzzing starts from, and the one
OSS-Fuzz builds. The targets are thin wrappers over the shared library.
Like cargo-fuzz's own template, the crate is a workspace of its own.
cargo fuzz run --fuzz-dir fuzz/libfuzzer fuzz_lattice_laws
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Build the libFuzzer targets with cargo-fuzz in a derivation, and add an
app that runs one of them against a corpus in the checkout, in the
layout cargo fuzz run uses:
nix run .#fuzz-libfuzzer -- fuzz_lattice_laws -max_total_time=600
Only the corpus lives outside the store, so a run does not depend on
whatever compiler and libraries the calling shell has.
The check, in the nightly aggregate, builds the targets with crash_test's
panic compiled in and confirms libFuzzer finds it within a minute. A
fuzzer with broken coverage instrumentation runs just as cleanly as one
with nothing to find, so the check asserts that a crash is found, not
that none is.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Whether libFuzzer solves crash_test within its time limit depends on its
random choices, so fuzz-libfuzzer-verify could pass or fail with no change
to its inputs. A check that flakes like that blames whichever commit
happens to be built, and a hard-to-find regression would point at the
wrong one.
Run it as an app instead, so CI can run it on a schedule:
nix run .#fuzz-libfuzzer-verify
The time limit goes from 60s to 600s, since an app is not bounded by a
check's budget, and libFuzzer arguments after -- override it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Once a day, confirm libFuzzer still finds the planted crash_test panic, then fuzz the lattice laws for half an hour. If either step fails, upload the crash artifacts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add a fuzz_scrub bin for the shared scrub target, and fuzz it on the schedule too. The workflow runs each target as its own matrix job, so a crash in one does not stop the other, and their artifacts stay apart. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds a cargo-fuzz frontend over the shared targets and runs it through nix:
nix run .#fuzz-libfuzzer -- fuzz_lattice_laws -max_total_time=600
The targets are built in a derivation and the app only runs them against
a corpus in the checkout.
Fuzzing is randomized, so it runs from a daily workflow rather than as a
flake check. That workflow first runs fuzz-libfuzzer-verify, which
confirms libFuzzer finds the crash_test panic.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3e7c10e to
f23a148
Compare
9f7f89b to
3d6bfa8
Compare
range-diff for force push 9f7f89b182 → 3d6bfa890a
|
range-diff for force push 9f7f89b182 → 3d6bfa890a
|
|
🤖 The first libFuzzer run of How it surfaced. One of the target's properties (6e6998a) is that whenever psbt-v2 accepts a PSBT, it also accepts the scrubbed form of that PSBT. Within seconds, libFuzzer produced a 46-byte input that breaks this. It decodes to a v2 PSBT with no inputs and one output, whose output map has these pairs, in this order:
Why. This is #280's mechanism. psbt-v2 0.3.0 treats an empty
So psbt-v2's verdict depends on the order of the pairs. The same pattern applies to What changed here. This wasn't a scrubber bug. The target now gives psbt-v2 the input with each map sorted and deduplicated, the way the scrubber writes it, so the reference parser sees the same pair order on both sides. After that change, a 10-minute libFuzzer run (4.4M executions) and a 5-minute AFL++ run (3.4M executions) found nothing. The sorting can stay even after we move to a psbt-v2 release that includes the fix. |
| # This is an app rather than a check: whether the fuzzer finds the crash | ||
| # within the time limit depends on its random choices, and a check that | ||
| # can fail without a change to its inputs would blame the wrong commit. | ||
| apps.fuzz-libfuzzer-verify = |
There was a problem hiding this comment.
PR CI does not compile the libfuzzer fuzz targets
No checks.* output builds fuzz/libfuzzer, so nix flake check and the per-commit validate-commits run never compile the new workspace; a type or API break that rustfmt does not catch can pass PR CI.
oak · automated · 3d6bfa8
bc1cindy
left a comment
There was a problem hiding this comment.
at 3d6bfa8 a 1 h fuzz_scrub run (71M execs) still breaks the accepts-implies-scrub-accepts property: 93 inputs have trailing bytes after the declared maps (psbt-v2 ignores them, scrub rejects), 28 have a 0-input tx, see #84 comment
fuzz_lattice_laws: 539M execs, nothing
minimal trailing-bytes case and repro
a valid v0 PSBT plus one 00:
70736274ff010033020000000100000000000000000000000000000000000000000000000000000000000000000000000000ffffffff0000000000000000
nix run .#fuzz-libfuzzer -- fuzz_scrub -fork=6 -ignore_crashes=1 -max_total_time=3600
Adds a cargo-fuzz frontend over the shared targets and runs it through nix:
The targets are built in a derivation and the app only runs them against
a corpus in the checkout.
There is a binary per target, fuzz_lattice_laws and fuzz_scrub, and the
workflow fuzzes each in a job of its own.
Fuzzing is randomized, so it runs from a daily workflow rather than as a
flake check. That workflow first runs fuzz-libfuzzer-verify, which
confirms libFuzzer finds the crash_test panic.
🤖 Generated with Claude Code