Repository navigation
fuzz with honggfuzz - #86
yuval-block wants to merge 6 commits into
Conversation
9514746 to
2c10c50
Compare
range-diff for force push 9514746e8f → 2c10c50741
|
honggfuzz mutates differently from libFuzzer and, on Linux, can use
hardware branch tracing for feedback, so it explores inputs libFuzzer's
corpus would not reach. The targets wrap the same shared library, in a
workspace of their own.
cd fuzz/honggfuzz && cargo hfuzz run hfuzz_lattice_laws
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Package cargo-hfuzz, which nixpkgs lacks. Build the honggfuzz targets
and the honggfuzz binary with it in a derivation, and add an app that
runs a target in persistent mode against a workspace in the checkout,
in the layout cargo hfuzz run uses:
nix run .#fuzz-honggfuzz -- hfuzz_lattice_laws --run_time 600
Only the workspace lives outside the store, so a run does not depend on
whatever compiler and libraries the calling shell has.
cargo-hfuzz compiles honggfuzz from C, so the build provides libbfd and
libunwind, gives the binary a runpath to them, and works around a clash
between the hardening wrapper's _FORTIFY_SOURCE and glibc's.
The check, in the nightly aggregate, builds the targets with crash_test's
panic compiled in and confirms honggfuzz finds it within a minute, which
it only does with working coverage feedback.
honggfuzz's Makefile supports macOS only up to 13 and builds through
xcrun, so all of this is Linux-only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Whether honggfuzz solves crash_test within its time limit depends on its
random choices, so fuzz-honggfuzz-verify could pass or fail with no change
to its inputs. A check that flakes like that blames whichever commit
happens to be built, and a hard-to-find regression would point at the
wrong one.
Run it as an app instead, so CI can run it on a schedule:
nix run .#fuzz-honggfuzz-verify
The time limit goes from 60s to 600s, since an app is not bounded by a
check's budget, and honggfuzz arguments after -- override it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Once a day, confirm honggfuzz still finds the planted crash_test panic, then fuzz the lattice laws for half an hour. honggfuzz exits cleanly even when it finds crashes, so a separate step fails the run if any were saved, and uploads them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add an hfuzz_scrub bin for the shared scrub target, and fuzz it on the schedule too. The workflow runs each target as its own matrix job, so a crash in one does not stop the other, and their crashes stay apart. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds a cargo-hfuzz frontend over the shared targets and runs it through nix:
nix run .#fuzz-honggfuzz -- hfuzz_lattice_laws --run_time 600
The targets and the honggfuzz binary are built in a derivation and the
app only runs them against a workspace in the checkout.
Fuzzing is randomized, so it runs from a daily workflow rather than as a
flake check. That workflow first runs fuzz-honggfuzz-verify, which
confirms honggfuzz finds the crash_test panic.
All of it is Linux-only: honggfuzz's Makefile supports macOS only up to 13.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| ]; | ||
|
|
||
| # Every target in fuzz/honggfuzz, and the honggfuzz binary itself, | ||
| # built by cargo-hfuzz in the sandbox so that the app and the check run |
There was a problem hiding this comment.
mkTargets comment still describes a check
The mkTargets comment still says a check runs the same binaries, but the verification is now an app that builds verifyTargets with RUSTFLAGS = "--cfg fuzzer_verify_crash"; the app and the verify app do not run the same binaries.
oak · automated · 2c10c50
| ${verifyTargets}/bin/honggfuzz -W "$scratch/workspace" -f "$scratch/input" -P \ | ||
| --timeout 5 --threads 1 --run_time 600 --exit_upon_crash "$@" \ | ||
| -- ${verifyTargets}/bin/hfuzz_crash_test || true | ||
| crashes=("$scratch"/workspace/*.fuzz) |
There was a problem hiding this comment.
Verify app accepts any crash
The verify app only checks that some .fuzz file exists, so an unrelated crash passes and it does not assert the crash_test panic the comment promises.
oak · automated · 2c10c50
| shift | ||
| # The same layout and persistent mode cargo hfuzz run uses. | ||
| workspace="fuzz/honggfuzz/hfuzz_workspace/$target" | ||
| mkdir -p "$workspace/input" |
There was a problem hiding this comment.
Fuzz app wrapper resolves mkdir from caller's PATH
The fuzz-honggfuzz app wrapper has no runtimeInputs, so mkdir comes from the caller's PATH and fails with mkdir: command not found when PATH lacks coreutils; should it declare coreutils like the verify app does?
oak · automated · 2c10c50
3e7c10e to
f23a148
Compare
2c10c50 to
6e422c9
Compare
range-diff for force push 2c10c50741 → 6e422c975c
|
range-diff for force push 2c10c50741 → 6e422c975c
|
| # A crash in one target says nothing about the others. | ||
| fail-fast: false | ||
| matrix: | ||
| target: [hfuzz_lattice_laws, hfuzz_scrub] |
There was a problem hiding this comment.
Verify step repeats in every matrix leg
The job-level matrix makes the target-independent verify step run in every leg; it always fuzzes hfuzz_crash_test with a hardcoded --run_time 600, so each scheduled run repeats the same up-to-600-second check. Move the verify step to a separate job or gate it to one matrix leg.
- name: Verify that honggfuzz finds the planted crash
— fungi-protocol/concurrent-psbt@ac9b9c0 · .github/workflows/fuzz-honggfuzz.yml#L31
--timeout 5 --threads 1 --run_time 600 --exit_upon_crash "$@" \
— fungi-protocol/concurrent-psbt@ac9b9c0 · nix/fuzz/honggfuzz.nix#L129
oak · automated · 6e422c9
| } | ||
| ); | ||
|
|
||
| targets = mkTargets { pnameSuffix = "-fuzz-honggfuzz"; }; |
There was a problem hiding this comment.
Package built with host-specific CPU features
cargo hfuzz build appends -C target-cpu=native while mkTargets leaves RUSTFLAGS empty, so packages.fuzz-honggfuzz is tuned to the build host. Because the derivation's output path is host-independent, a cached binary may use instructions the user's CPU lacks.
oak · automated · 6e422c9
| # crash within the time limit depends on its random choices, and a | ||
| # check that can fail without a change to its inputs would blame the | ||
| # wrong commit. | ||
| fuzz-honggfuzz-verify = |
There was a problem hiding this comment.
Honggfuzz frontend compile errors can merge
fuzz-honggfuzz-verify is an app, not a check, and PR CI's nix flake check does not build app programs; no check builds fuzz/honggfuzz, so a compile error can merge and surface only in the main-only packages build or the scheduled fuzz workflow.
oak · automated · 6e422c9
There was a problem hiding this comment.
works on x86_64 Linux, verify found crash_test in 17 s
1 h each: hfuzz_scrub (72M execs) hit one crash signature, the "psbt-v2 accepts, scrub rejects" check from #84; its ~13.5k timeouts are likely that crash too, since apport makes each crash take ~1.2 s on this host (slowest_unit_ms:1224), over the 1 s timeout. hfuzz_lattice_laws (637M execs) found nothing, with 4 timeouts
honggfuzz summaries
at 6e422c9, nix run .#fuzz-honggfuzz-verify -- --threads 6 and nix run .#fuzz-honggfuzz -- <target> --threads 6 --run_time 3600
verify: iterations:5156413 time:17 crashes_count:1 timeout_count:0
scrub: iterations:71929548 time:3601 crashes_count:13503 timeout_count:13497
lattice: iterations:636998589 time:3601 crashes_count:0 timeout_count:4
Adds a cargo-hfuzz frontend over the shared targets and runs it through nix:
The targets and the honggfuzz binary are built in a derivation and the
app only runs them against a workspace in the checkout.
There is a binary per target, hfuzz_lattice_laws and hfuzz_scrub, and the
workflow fuzzes each in a job of its own.
Fuzzing is randomized, so it runs from a daily workflow rather than as a
flake check. That workflow first runs fuzz-honggfuzz-verify, which
confirms honggfuzz finds the crash_test panic.
All of it is Linux-only: honggfuzz's Makefile supports macOS only up to 13.
🤖 Generated with Claude Code