Skip to content

The audit event model.integrity_verified records a verification that never happens (CR-A02-091) #380

Open

Description

@cemililik

Wave 1 · theme D1-2: Audit chain, integrity manifest and verifier robustness · highest severity High · 2 finding(s) · effort small

Summary

An auditor reads model.integrity_verified in audit_log.jsonl as evidence that the shipped directory was verified, but it was only hashed.

Primary finding

CR-A02-091 — The audit event model.integrity_verified records a verification that never happens

severity Low · reachability default · intent contract-violation · confidence likely · single reviewer · not reproduced (code trace)

What happens. _generate_model_integrity emits model.integrity_verified right after writing the manifest (trainer.py:2167). The audit event catalog defines the event as 'written and re-hashed successfully after training' (audit_event_catalog.md:60). No re-hash or verify_integrity call takes place. Combined with the deployer-instructions ordering defect, the Art. 12 log records 'integrity verified' for a directory that verify-integrity rejects with exit 6. This affects provenance accuracy.

Where.

Evidence.

  • trainer.py:2162-2167: generate, write, then log_event('model.integrity_verified', artifacts=len(...)). verify_integrity is never called.

Recommended fix. Call verify_integrity after the final write and log its outcome, or change the catalog text to say 'manifest generated'. A rename of the event is breaking.

Other findings with the same root cause

These close with the same fix; update them in the same pull request.

DOC-D1B04-004 — docs/reference/audit_event_catalog.md: false claim — model.integrity_verified is emitted when model_integrity.json 'was written and re-hashed successfully after training'; artifacts is the count of…

severity High

Where. docs/reference/audit_event_catalog.md:60 (Turkish mirror: docs/reference/audit_event_catalog-tr.md:60) · type: false claim

The document says. model.integrity_verified is emitted when model_integrity.json 'was written and re-hashed successfully after training'; artifacts is the count of files re-hashed.

What the code does. The trainer hashes the files once with generate_model_integrity, writes the JSON and emits the event; nothing is re-hashed or verified. deployer_instructions.md is then written into the same directory, so the manifest is already incomplete when the event claims verification.

Code references. forgelm/trainer.py:2159-2167, forgelm/trainer.py:1503-1506, forgelm/compliance.py:1761

User impact. The Art. 15 record claims a verification that never happened, and the manifest it refers to fails verify-integrity (exit 6).

Proposed correction. Fix the code: generate model_integrity.json after every writer into final_path, then run forgelm.verify.verify_integrity before emitting the event. Until then: 'Emitted after model_integrity.json is written (hashes computed once; not re-verified).'

Turkish mirror. the Turkish page has no such statement

Acceptance criteria

  • A regression test reproduces the failure scenario and asserts the corrected behaviour.
  • Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
  • Every finding in this issue is resolved or explicitly closed with a reason.

Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: trainerCode area: trainerbugSomething isn't workingdocumentationImprovements or additions to documentationseverity: highWrong behaviour or a false claim users rely onsource: review-2026-10Found by the October 2026 code and documentation reviewwave: 1Roadmap wave 1: high severity, small effort

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions