Repository navigation
[Phase 16] S5 — Verification truth: three states, not a success bit #891
Description
Roadmap · Phase 16, step S5 · size L · 3 unit(s) · planned in
docs/roadmap/phase-16-trust-surface-hardening.md:416-447
Summary
Three verifiers project a three-valued reality (verified / unverified / invalid) onto a two-valued valid plus an exit code, so "no strong layer ran" shares a success bit with "every strong layer passed". The exporter never writes a .sha256 sidecar for GGUF and gguf is in no dependency group, so a magic-only valid=true is the default outcome for ForgeLM's own artefacts. The writer half (CLI-03): export records its hash under exported_artifacts, a key the verifier never reads — it fails closed, a round-trip-integrity defect rather than a false pass.
Units
| Unit | Tracked by (October 2026 review) |
|---|---|
F-W20260729-TRUST-04 |
#83 · #84 · #85 · #446 · #566 · #573 · #598 · #599 |
F-W20260729-TRUST-05 |
#164 · #328 · #364 · #558 · #658 |
F-W20260729-CLI-03 |
#84 · #86 · #567 · #621 · #652 · #766 |
Decisions this step executes
- C-3 —
verify-ggufmagic-only verdict: opt-in--require-strong-layer; the default flip waits for the next MAJOR (tracked as a deferred issue). - C-4 —
forgelm exportwrites the.sha256sidecar its docs already claim.
Plan
- Add
checks['strong_layer_ran']and the opt-in--require-strong-layerflag. - Make
forgelm exportwrite the GGUF sidecar, so the strong layer is present by default. - Also owned by this step (deferral row
F-W20260729-VERIFY-UNCAPPED-READ): route the two unboundedjson.loadreads inverify_annex_iv_artifactandverify_integritythrough_io_safety._read_capped_json, so a deeply nested payload cannot end inRecursionError.
Exit criteria
- A 4-byte
b"GGUF"file with no parser and no sidecar does not report an unqualified success. - Hash-less Annex IV and pipeline roots do not return exit 0 by default, while any legacy override still reports
verified: false. - An export → tamper → verify round trip exits 6 on a flipped byte.
- The existing 1-vs-6 split is preserved exactly and stays structural (typed fields, never
reasonprose); the per-stageUNVERIFIEDbehaviour is asserted unchanged. - Both verifiers parse operator-supplied JSON through the capped reader.
How to deliver
- One root-cause family, committed on its own; then an Opus review round and a Sonnet review round, each with verified findings fixed and committed before the next step begins.
- A reproducer that is red before the fix and green after; the full gauntlet from
CLAUDE.mdpasses under./.venv/bin/python. - Every English documentation edit ships with its Turkish mirror in the same commit; a new or renamed audit event gets its EN and TR catalog rows in the same PR.
- A step that grows a deferred module carries its
budget_historyjustification in the same diff; a widened guard rolls out non-strict first, then flips. - Commit bodies cite
Absorbs F-W20260729-…; the CHANGELOG[Unreleased]section gets an entry only for a change a user or library consumer can observe (decision C-20).
Planned in docs/roadmap/phase-16-trust-surface-hardening.md (read at f94595f). Unit IDs (F-W20260729-…) come from the 2026-07-29/30 full-project review; the issue numbers next to them are the October 2026 review's records of the same defects.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsTodo
Activity
forgelm/cli/_parser.pyinto a sub-package (_train,_inspect,_data,_run) — deferred until Phase 16 S5, S7 and S11 have landed #907