Skip to content

fuzz with honggfuzz - #86

Draft
yuval-block wants to merge 6 commits into
fuzz-targetsfrom
fuzz-honggfuzz
Draft

yuval-block wants to merge 6 commits into
fuzz-targetsfrom
fuzz-honggfuzz

Conversation

@yuval-block

@yuval-block yuval-block commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Adds a cargo-hfuzz frontend over the shared targets and runs it through nix:

nix run .#fuzz-honggfuzz -- hfuzz_lattice_laws --run_time 600

The targets and the honggfuzz binary are built in a derivation and the
app only runs them against a workspace in the checkout.

There is a binary per target, hfuzz_lattice_laws and hfuzz_scrub, and the
workflow fuzzes each in a job of its own.

Fuzzing is randomized, so it runs from a daily workflow rather than as a
flake check. That workflow first runs fuzz-honggfuzz-verify, which
confirms honggfuzz finds the crash_test panic.

All of it is Linux-only: honggfuzz's Makefile supports macOS only up to 13.

🤖 Generated with Claude Code

@yuval-block yuval-block mentioned this pull request Nov 12, 2026
@github-code-quality

github-code-quality Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Code coverage overview

Languages: Rust

Rust / code-coverage/cargo-llvm-cov

fuzz-targets 227c7b5 fuzz-honggfuzz 6e422c9 +/-
100% 100% 0%

ℹ️ Calculations represent line coverage. Results may change as calculations are updated.


Updated November 13, 2026 06:44 UTC

@github-actions

Copy link
Copy Markdown
range-diff for force push 9514746e8f → 2c10c50741 November 13, 2026 03:21 UTC: 2 unchanged, 1 dropped, 3 added; 3 → 5 commits; same base 3e7c10e5fa

reproduce: git range-diff 3e7c10e..9514746 3e7c10e..2c10c50

1:  4252e751cb +344  -0 5f = 1:  4252e751cb +344  -0 5f fuzz: honggfuzz frontend
2:  7d695a75ef +131  -0 1f = 2:  7d695a75ef +131  -0 1f nix: fuzz with honggfuzz
3:  9514746e8f   +0  -0 0f < -:  ----------             fuzz with honggfuzz
-:  ----------             > 3:  9e56e052f5  +32 -21 1f nix: verify honggfuzz from an app, not a check
-:  ----------             > 4:  03b6f83d13  +42  -0 1f ci: fuzz with honggfuzz on a schedule
-:  ----------             > 5:  2c10c50741   +0  -0 0f fuzz with honggfuzz

nothingmuch and others added 6 commits November 13, 2026 06:50
honggfuzz mutates differently from libFuzzer and, on Linux, can use
hardware branch tracing for feedback, so it explores inputs libFuzzer's
corpus would not reach. The targets wrap the same shared library, in a
workspace of their own.

    cd fuzz/honggfuzz && cargo hfuzz run hfuzz_lattice_laws

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Package cargo-hfuzz, which nixpkgs lacks. Build the honggfuzz targets
and the honggfuzz binary with it in a derivation, and add an app that
runs a target in persistent mode against a workspace in the checkout,
in the layout cargo hfuzz run uses:

    nix run .#fuzz-honggfuzz -- hfuzz_lattice_laws --run_time 600

Only the workspace lives outside the store, so a run does not depend on
whatever compiler and libraries the calling shell has.

cargo-hfuzz compiles honggfuzz from C, so the build provides libbfd and
libunwind, gives the binary a runpath to them, and works around a clash
between the hardening wrapper's _FORTIFY_SOURCE and glibc's.

The check, in the nightly aggregate, builds the targets with crash_test's
panic compiled in and confirms honggfuzz finds it within a minute, which
it only does with working coverage feedback.

honggfuzz's Makefile supports macOS only up to 13 and builds through
xcrun, so all of this is Linux-only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Whether honggfuzz solves crash_test within its time limit depends on its
random choices, so fuzz-honggfuzz-verify could pass or fail with no change
to its inputs. A check that flakes like that blames whichever commit
happens to be built, and a hard-to-find regression would point at the
wrong one.

Run it as an app instead, so CI can run it on a schedule:

    nix run .#fuzz-honggfuzz-verify

The time limit goes from 60s to 600s, since an app is not bounded by a
check's budget, and honggfuzz arguments after -- override it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Once a day, confirm honggfuzz still finds the planted crash_test panic,
then fuzz the lattice laws for half an hour. honggfuzz exits cleanly even
when it finds crashes, so a separate step fails the run if any were
saved, and uploads them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add an hfuzz_scrub bin for the shared scrub target, and fuzz it on the
schedule too. The workflow runs each target as its own matrix job, so a
crash in one does not stop the other, and their crashes stay apart.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds a cargo-hfuzz frontend over the shared targets and runs it through nix:

    nix run .#fuzz-honggfuzz -- hfuzz_lattice_laws --run_time 600

The targets and the honggfuzz binary are built in a derivation and the
app only runs them against a workspace in the checkout.

Fuzzing is randomized, so it runs from a daily workflow rather than as a
flake check. That workflow first runs fuzz-honggfuzz-verify, which
confirms honggfuzz finds the crash_test panic.

All of it is Linux-only: honggfuzz's Makefile supports macOS only up to 13.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread nix/fuzz/honggfuzz.nix
];

# Every target in fuzz/honggfuzz, and the honggfuzz binary itself,
# built by cargo-hfuzz in the sandbox so that the app and the check run

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mkTargets comment still describes a check

The mkTargets comment still says a check runs the same binaries, but the verification is now an app that builds verifyTargets with RUSTFLAGS = "--cfg fuzzer_verify_crash"; the app and the verify app do not run the same binaries.

oak · automated · 2c10c50

Comment thread nix/fuzz/honggfuzz.nix
${verifyTargets}/bin/honggfuzz -W "$scratch/workspace" -f "$scratch/input" -P \
--timeout 5 --threads 1 --run_time 600 --exit_upon_crash "$@" \
-- ${verifyTargets}/bin/hfuzz_crash_test || true
crashes=("$scratch"/workspace/*.fuzz)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verify app accepts any crash

The verify app only checks that some .fuzz file exists, so an unrelated crash passes and it does not assert the crash_test panic the comment promises.

oak · automated · 2c10c50

Comment thread nix/fuzz/honggfuzz.nix
shift
# The same layout and persistent mode cargo hfuzz run uses.
workspace="fuzz/honggfuzz/hfuzz_workspace/$target"
mkdir -p "$workspace/input"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fuzz app wrapper resolves mkdir from caller's PATH

The fuzz-honggfuzz app wrapper has no runtimeInputs, so mkdir comes from the caller's PATH and fails with mkdir: command not found when PATH lacks coreutils; should it declare coreutils like the verify app does?

oak · automated · 2c10c50

@github-actions

Copy link
Copy Markdown
range-diff for force push 2c10c50741 → 6e422c975c November 13, 2026 06:29 UTC: 5 unchanged, 1 added; 5 → 6 commits; base 3e7c10e5fa → f23a1484f3 (diverged, 2 ahead and 1 behind)

reproduce: git range-diff 3e7c10e..2c10c50 f23a148..6e422c9

1:  4252e751cb +344  -0 5f = 1:  2d398c0a98 +344  -0 5f fuzz: honggfuzz frontend
2:  7d695a75ef +131  -0 1f = 2:  6cf0ebfaea +131  -0 1f nix: fuzz with honggfuzz
3:  9e56e052f5  +32 -21 1f = 3:  4f6ed04aaf  +32 -21 1f nix: verify honggfuzz from an app, not a check
4:  03b6f83d13  +42  -0 1f = 4:  dabab79ffb  +42  -0 1f ci: fuzz with honggfuzz on a schedule
-:  ----------             > 5:  ac9b9c02f2  +27  -6 3f fuzz: scrub with honggfuzz
5:  2c10c50741   +0  -0 0f = 6:  6e422c975c   +0  -0 0f fuzz with honggfuzz

@github-actions

Copy link
Copy Markdown
range-diff for force push 2c10c50741 → 6e422c975c November 13, 2026 06:29 UTC: 5 unchanged, 1 added; 5 → 6 commits; base 3e7c10e5fa → f23a1484f3 (diverged, 2 ahead and 1 behind)

reproduce: git range-diff 3e7c10e..2c10c50 f23a148..6e422c9

1:  4252e751cb +344  -0 5f = 1:  2d398c0a98 +344  -0 5f fuzz: honggfuzz frontend
2:  7d695a75ef +131  -0 1f = 2:  6cf0ebfaea +131  -0 1f nix: fuzz with honggfuzz
3:  9e56e052f5  +32 -21 1f = 3:  4f6ed04aaf  +32 -21 1f nix: verify honggfuzz from an app, not a check
4:  03b6f83d13  +42  -0 1f = 4:  dabab79ffb  +42  -0 1f ci: fuzz with honggfuzz on a schedule
-:  ----------             > 5:  ac9b9c02f2  +27  -6 3f fuzz: scrub with honggfuzz
5:  2c10c50741   +0  -0 0f = 6:  6e422c975c   +0  -0 0f fuzz with honggfuzz

# A crash in one target says nothing about the others.
fail-fast: false
matrix:
target: [hfuzz_lattice_laws, hfuzz_scrub]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verify step repeats in every matrix leg

The job-level matrix makes the target-independent verify step run in every leg; it always fuzzes hfuzz_crash_test with a hardcoded --run_time 600, so each scheduled run repeats the same up-to-600-second check. Move the verify step to a separate job or gate it to one matrix leg.

  • name: Verify that honggfuzz finds the planted crash

— fungi-protocol/concurrent-psbt@ac9b9c0 · .github/workflows/fuzz-honggfuzz.yml#L31

--timeout 5 --threads 1 --run_time 600 --exit_upon_crash "$@" \

— fungi-protocol/concurrent-psbt@ac9b9c0 · nix/fuzz/honggfuzz.nix#L129

oak · automated · 6e422c9

Comment thread nix/fuzz/honggfuzz.nix
}
);

targets = mkTargets { pnameSuffix = "-fuzz-honggfuzz"; };

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Package built with host-specific CPU features

cargo hfuzz build appends -C target-cpu=native while mkTargets leaves RUSTFLAGS empty, so packages.fuzz-honggfuzz is tuned to the build host. Because the derivation's output path is host-independent, a cached binary may use instructions the user's CPU lacks.

oak · automated · 6e422c9

Comment thread nix/fuzz/honggfuzz.nix
# crash within the time limit depends on its random choices, and a
# check that can fail without a change to its inputs would blame the
# wrong commit.
fuzz-honggfuzz-verify =

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honggfuzz frontend compile errors can merge

fuzz-honggfuzz-verify is an app, not a check, and PR CI's nix flake check does not build app programs; no check builds fuzz/honggfuzz, so a compile error can merge and surface only in the main-only packages build or the scheduled fuzz workflow.

oak · automated · 6e422c9

@bc1cindy bc1cindy left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

works on x86_64 Linux, verify found crash_test in 17 s

1 h each: hfuzz_scrub (72M execs) hit one crash signature, the "psbt-v2 accepts, scrub rejects" check from #84; its ~13.5k timeouts are likely that crash too, since apport makes each crash take ~1.2 s on this host (slowest_unit_ms:1224), over the 1 s timeout. hfuzz_lattice_laws (637M execs) found nothing, with 4 timeouts

honggfuzz summaries

at 6e422c9, nix run .#fuzz-honggfuzz-verify -- --threads 6 and nix run .#fuzz-honggfuzz -- <target> --threads 6 --run_time 3600

verify:  iterations:5156413 time:17 crashes_count:1 timeout_count:0
scrub:   iterations:71929548 time:3601 crashes_count:13503 timeout_count:13497
lattice: iterations:636998589 time:3601 crashes_count:0 timeout_count:4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants