Skip to content

[Phase 16] S5 — Verification truth: three states, not a success bit #891

Open

Description

@cemililik

Roadmap · Phase 16, step S5 · size L · 3 unit(s) · planned in docs/roadmap/phase-16-trust-surface-hardening.md:416-447

Summary

Three verifiers project a three-valued reality (verified / unverified / invalid) onto a two-valued valid plus an exit code, so "no strong layer ran" shares a success bit with "every strong layer passed". The exporter never writes a .sha256 sidecar for GGUF and gguf is in no dependency group, so a magic-only valid=true is the default outcome for ForgeLM's own artefacts. The writer half (CLI-03): export records its hash under exported_artifacts, a key the verifier never reads — it fails closed, a round-trip-integrity defect rather than a false pass.

Units

Unit Tracked by (October 2026 review)
F-W20260729-TRUST-04 #83 · #84 · #85 · #446 · #566 · #573 · #598 · #599
F-W20260729-TRUST-05 #164 · #328 · #364 · #558 · #658
F-W20260729-CLI-03 #84 · #86 · #567 · #621 · #652 · #766

Decisions this step executes

  • C-3 — verify-gguf magic-only verdict: opt-in --require-strong-layer; the default flip waits for the next MAJOR (tracked as a deferred issue).
  • C-4 — forgelm export writes the .sha256 sidecar its docs already claim.

Plan

  • Add checks['strong_layer_ran'] and the opt-in --require-strong-layer flag.
  • Make forgelm export write the GGUF sidecar, so the strong layer is present by default.
  • Also owned by this step (deferral row F-W20260729-VERIFY-UNCAPPED-READ): route the two unbounded json.load reads in verify_annex_iv_artifact and verify_integrity through _io_safety._read_capped_json, so a deeply nested payload cannot end in RecursionError.

Exit criteria

  • A 4-byte b"GGUF" file with no parser and no sidecar does not report an unqualified success.
  • Hash-less Annex IV and pipeline roots do not return exit 0 by default, while any legacy override still reports verified: false.
  • An export → tamper → verify round trip exits 6 on a flipped byte.
  • The existing 1-vs-6 split is preserved exactly and stays structural (typed fields, never reason prose); the per-stage UNVERIFIED behaviour is asserted unchanged.
  • Both verifiers parse operator-supplied JSON through the capped reader.

How to deliver

  • One root-cause family, committed on its own; then an Opus review round and a Sonnet review round, each with verified findings fixed and committed before the next step begins.
  • A reproducer that is red before the fix and green after; the full gauntlet from CLAUDE.md passes under ./.venv/bin/python.
  • Every English documentation edit ships with its Turkish mirror in the same commit; a new or renamed audit event gets its EN and TR catalog rows in the same PR.
  • A step that grows a deferred module carries its budget_history justification in the same diff; a widened guard rolls out non-strict first, then flips.
  • Commit bodies cite Absorbs F-W20260729-…; the CHANGELOG [Unreleased] section gets an entry only for a change a user or library consumer can observe (decision C-20).

Planned in docs/roadmap/phase-16-trust-surface-hardening.md (read at f94595f). Unit IDs (F-W20260729-…) come from the 2026-07-29/30 full-project review; the issue numbers next to them are the October 2026 review's records of the same defects.

Activity

  1. added
    phase: 16Roadmap Phase 16: trust surface hardening (docs/roadmap/phase-16-trust-surface-hardening.md)
    source: roadmapPlanned or deferred work recorded in docs/roadmap/
    on Oct 9, 2026
  2. added
    blockedWaits for another issue (see Blocked by); set and removed automatically, never by hand
    on Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    blockedWaits for another issue (see Blocked by); set and removed automatically, never by handphase: 16Roadmap Phase 16: trust surface hardening (docs/roadmap/phase-16-trust-surface-hardening.md)source: roadmapPlanned or deferred work recorded in docs/roadmap/

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions