ci: fuzz for real with Jazzer.js; cflite.yml is the coverage report it always was - #122
Conversation
sprayberry-redline
left a comment
There was a problem hiding this comment.
Automated review from the Sprayberry Labs fleet code reviewer.
Reviewed by the gating lane (gating review).
Verdict: request changes. One blocking finding: fuzz.yml's pull_request path filter lists scripts/** and policy/**, which the fuzz targets never import, and omits cdp-proxy.mjs and ua.mjs, the two root modules the targets actually exercise, so a PR changing the fuzzed guards does not run the fuzzer despite the header's promise. Checked the runner's Jazzer invocation (positional corpus dir, artifact_prefix with trailing separator, cwd-relative target path unchanged from before), the cache restore/save key scheme, the FUZZ_SECONDS expression for each trigger, action pinning and permissions, the fork guard, the fleet-status list update, the cflite.yml header rewrite, and the public text for generated-sounding prose or attribution; all fine.
1. Blocking: .github/workflows/fuzz.yml:21
pull_request:
paths:
- 'fuzz/'
- 'scripts/'
- 'policy/**'
- 'package.json'
- 'package-lock.json'
- '.github/workflows/fuzz.yml'
The targets import '../cdp-proxy.mjs' (fuzz/cdp_guards.fuzz.js:18) and '../ua.mjs' (fuzz/ua.fuzz.js:14), both at the repository root, and those modules import only node built-ins. Neither scripts/ nor policy/ is reachable from any target. A pull request that edits hostIsIpOrLocalhost or strippedPath in cdp-proxy.mjs, or pickUa in ua.mjs, does not match any listed path, so the fuzzer does not run on exactly the changes the header and PR description say it guards ("on pull requests that touch the targets or the code under them"). The filter appears carried over from plumbline#60 rather than derived from this repository's targets.
Suggested fix:
Replace 'scripts/**' and 'policy/**' with 'cdp-proxy.mjs' and 'ua.mjs' (or list the root .mjs modules the targets import). Keep fuzz/**, package.json, package-lock.json and the workflow file.
rule:wrong-trigger-paths
sprayberry-redline
left a comment
There was a problem hiding this comment.
Automated review from the Sprayberry Labs fleet code reviewer.
Reviewed by the gating lane (gating review).
Verdict: approve. Approving. I checked fuzz.yml (read-only contents permission, all actions SHA-pinned, the workflow_dispatch input only reaches libFuzzer as an argv element with no shell, cache restore-by-prefix/save-by-run_id is sound and PR runs read the default-branch corpus without writing it), fuzz/run.mjs (Jazzer CLI positional corpus dir after the boolean --sync flag, artifact prefix placed after the -- separator, signal/ENOENT exits mapped to a non-zero status), the fleet-status list against the self-test in scripts/fleet-status.test.mjs (it keys on name: Fuzz, which matches), and the pull_request path filter against the targets' imports (cdp-proxy.mjs and ua.mjs have no local imports, so the filter is complete). The public text avoids generated-sounding filler and contains no attribution or secrets. One non-blocking note about README/docs text the PR's premise leaves stale.
Minor:
.github/workflows/fuzz.yml:4: README.md:41 ("ClusterFuzzLite runs two Jazzer.js targets weekly"), docs/releases.md:10 ("ClusterFuzzLite weekly") and the README badge labelled "ClusterFuzzLite" still describe cflite.yml as the fuzzer. With this change the repository's own workflow headers say the opposite, so a reader of the public docs gets a claim the CI config now explicitly disowns.
What
.github/workflows/fuzz.yml(new): every target infuzz/runs under Jazzer.js (libFuzzer) for 300 s weekly and on demand, 60 s on pull requests that touchfuzz/or the code under it. The corpus persists between runs through the Actions cache (restored by prefix, saved under the run id on the default branch). A crash, uncaught throw or timeout fails the job and the reproducing input is uploaded as an artifact.fuzz/run.mjs: the runner the workflow and a developer both use, withFUZZ_SECONDS,FUZZ_CORPUS_DIRandFUZZ_ARTIFACT_DIR; same Jazzer invocation as before, plus the corpus and artifact arguments..github/workflows/cflite.yml: renamed "ClusterFuzzLite coverage", job "Coverage report", header and matrix comment say what it does. Nothing else changes in it;.clusterfuzzlite/stays so Scorecard's Fuzzing check keeps its source..github/workflows/fleet-status.yml:Fuzzjoins theworkflow_runlist (its self-test requires every pull_request workflow to be listed)..gitignore: the corpus and crash dirs.Why
The ClusterFuzzLite action cannot fuzz JavaScript at all, so no run of cflite.yml has ever fuzzed anything: the OSS-Fuzz builder rejects every sanitizer for JS ("JavaScript projects cannot be fuzzed with sanitizers", plumbline run 36204398437), the action's config rejects
none("Invalid SANITIZER: none", run 36204619235), andcoverageselects the coverage-report runner (every past run here: "MODE is: coverage", a 3–5 s "Run fuzzers" step, "Contains 0 elements", only acifuzz-coverage-latestartifact). The workflow's old header promised continuous fuzzing with crashes failing the job; Scorecard credits Fuzzing 10/10 from the config. Same change as askalf/plumbline#60, whose pull-request run fuzzed three targets for real (36205040701).Proof
The pull request run of fuzz.yml on this branch is the test: 60 s per target, Jazzer's own output (INITED, NEW cov, REDUCE) for every target, corpus files written.