Repository navigation
[Wave 4] Low/info — Module sweep: Compliance core: Annex IV, audit log, integrity manifest, provenance (part 4/5) #659
Description
Wave 4 · 20 finding(s) in 20 fix package(s). Fix these whenever a pull request touches this module ("clean the module you touch").
Findings
-
CR-A07-129 (code, Low) — Line-model edge cases contradict the docs: CRLF conversion, LF->CR and blank-line insertion pass verify-audit --require-hmac although 'any byte-level corruption ... shows up as a chain break'; a BOM…
severity Low · reachabilitydefault· intentcontract-violation· confidence likely · independently confirmed · reproduced end to end
Failure scenario. An auditor relies on the manual's promise that any byte-level change in transit is a chain break, and treats a CRLF-converted copy as byte-identical evidence. Separately, an operator chasing approve's 'corrupted at line 2' opens line 2 of a log that verify-audit numbers differently.
Where.forgelm/compliance.py:2658-2660,forgelm/compliance.py:2542-2551,forgelm/compliance.py:288-294,docs/usermanuals/en/compliance/verify-audit.md:14
Recommended fix. Either make the verifier byte-exact (LF-only, reject \r and blank records, as the writer never emits them) or reword the manual to say that the chain authenticates record content, with line-ending and blank-line changes tolerated. Report physical file line numbers consistently across verify-audit and the approval readers. -
CR-A07-130 (code, Low) — access_control.md section 5: 'Audit logs in <output_dir> inherit the dir's perm' is wrong; the log and its genesis manifest are created with the umask mode (0644)
severity Low · reachabilitydefault· intentcontract-violation· confidence likely · single reviewer · reproduced end to end
Failure scenario. An archived output dir (section 3.4 tells operators to archive it) exposes a world-readable audit log that carries operator identifiers.
Where.docs/qms/access_control.md:172,docs/qms/access_control-tr.md:174,forgelm/compliance.py:549,forgelm/compliance.py:468
Recommended fix. Say that files are created with the process umask and that protection comes from the 0700 directory (or recommend umask 077), in EN and TR. -
CR-A07-131 (code, Low) — compliance_summary.md cites a non-existent function
compute_artefact_sha256as the Article 15 model-integrity mechanism
severity Low · reachabilityunreachable· intentcontract-violation· confidence likely · independently confirmed · reproduced end to end
Failure scenario. An auditor or integrator traces the Art. 15 claim tocompute_artefact_sha256and finds nothing, which weakens the traceability of the compliance summary.
Where.docs/reference/compliance_summary.md:35,docs/reference/compliance_summary-tr.md:33,forgelm/compliance.py:760
Recommended fix. Replace the name withgenerate_model_integrity/hash_filein both language versions. -
CR-A07-132 (code, Low) — The pipeline structural verifier accepts manifest shapes the writer can never produce: duplicate stage names (the first duplicate decides the stopped_at check), bool or float index, empty stages list
severity Low · reachabilitydefault· intentcontract-violation· confidence likely · independently confirmed · reproduced end to end
Failure scenario. A manifest that says the pipeline stopped at 'dpo' because 'dpo' failed, while also carrying a completed and deep-verified 'dpo' row, is reported OK with exit 0. Whether it passes depends only on which duplicate comes first.
Where.forgelm/compliance.py:2012-2019,forgelm/compliance.py:2057,forgelm/compliance.py:2092-2095,forgelm/compliance.py:2114-2116
Recommended fix. In _verify_manifest_payload: flag duplicate stage names, require type(index) is int (rejecting bool), and flag an empty stages list. Once names are unique, matching[0] is well defined. -
CR-A07-133 (code, Low) — Under an arbitrary passwd-less UID (docker run --user / OpenShift random UID), training aborts with ConfigError (exit 1) only after model load and dataset preparation; --dry-run passes and the contai…
severity Low · reachabilitydefault· intentintended-but-harmful· confidence likely · independently confirmed · reproduced with stubs
Failure scenario. A platform team deploys the image on OpenShift (random UID) with no FORGELM_OPERATOR. The CI dry-run step passes. The training job downloads and loads a 7B model and tokenises the dataset, then exits 1 with 'Operator identity unavailable' after minutes to tens of minutes of GPU time. The docs had said behaviour on Kubernetes is identical.
Where.forgelm/compliance.py:196-228,forgelm/trainer.py:354,forgelm/cli/_training.py:141-148,forgelm/cli/_training.py:163-170
Recommended fix. Resolve the operator identity before model load: construct the AuditLogger or call a shared resolver early in _run_training_pipeline, and include it in --dry-run. Document FORGELM_OPERATOR as required for non-default UIDs in docker.md and enterprise_deployment.md, and qualify the 'behaviour is identical' tip. -
CR-A07-134 (code, Info) — export_evidence_bundle (no production caller) zips its own growing .tmp when bundle_path is inside output_dir and follows symlinked files out of the tree
severity Info · reachabilityundocumented· intentintended-but-harmful· confidence likely · independently confirmed · reproduced end to end
Failure scenario. An integrator writes the bundle next to the artefacts and ships an archive that contains a truncated copy of itself and files from outside the compliance dir.
Where.forgelm/compliance.py:2315-2319(os.walk + zf.write with no exclusion and no realpath containment),docs/reference/audit_event_catalog.md:73,forgelm/compliance.py:2312,forgelm/compliance.py:2315
Recommended fix. Skip the tmp and bundle paths and refuse escaping symlinks (reuse the realpath containment from generate_model_integrity). Consider wiring a CLI entry or dropping the F-090 claim. -
CR-A07-135 (code, Info) — log_event creates audit_log.jsonl before serialising, so a failed first event leaves a zero-byte log that verify-audit says ForgeLM never writes
severity Info · reachabilityundocumented· intentintended-but-harmful· confidence confirmed · independently confirmed · reproduced end to end
Where.forgelm/compliance.py:549,forgelm/compliance.py:582,forgelm/compliance.py:2756-2763,forgelm/compliance.py:2719-2727
Recommended fix. Serialise the entry (and its HMAC) before opening the file, apart from prev_hash, which must be read under the lock. Alternatively, soften the verifier message so it does not assert impossibility. -
CR-A07-136 (code, Info) — Pipeline corroboration parses each audit line three times and holds about 4.4x the log size
severity Info · reachabilitydefault· intentintended-and-acceptable· confidence likely · single reviewer · reproduced end to end
Failure scenario. A 30 MB audit log (about 200k events) from a long multi-stage pipeline is verified with FORGELM_AUDIT_SECRET set. Peak traced memory is about 4.4x the file size and runtime is about 1.6 s per 22 MB. This is correct but wasteful. A log just over the cap fails as oversize (exit 1), so a log that has outgrown the cap cannot be corroborated at all, and no rotation or streaming path exists.
Where.forgelm/compliance.py:3365-3406,forgelm/compliance.py:3445-3456,forgelm/compliance.py:2542-2576,forgelm/compliance.py:2433-2438
Recommended fix. Parse each line once and pass the (line, entry) pairs into the chain walk. That would cut CPU by about 1.5 to 2x and remove the retained duplicate. The retained dicts could also be reduced to the few fields _live_passed_stages needs (event, pipeline_run_id, stage_name, gate_decision). Expected memory gain is about 3 to 4x. Raising the 32 MiB cap is not advisable until this is done. -
CR-A07-137 (code, Info) — export_evidence_bundle deflates incompressible weights with no size guard or caller in the codebase
severity Info · reachabilityundocumented· intentintended-and-acceptable· confidence likely · independently confirmed · reproduced end to end
Failure scenario. An integrator points export_evidence_bundle at a directory that holds a 14 GB model. Bundling takes about 3 minutes of single-core deflate and produces a 14 GB archive, where ZIP_STORED would run at disk speed.
Where.forgelm/compliance.py:2314-2319,docs/reference/architecture.md:147
Recommended fix. Use ZIP_STORED for known-incompressible suffixes (.safetensors, .bin, .gguf, .pt), or restrict the bundle to the compliance/ subtree. Expected gain is about 10x or more on weight-heavy trees. The first option is cheap, but check first that no caller exists, since the function appears dead. -
CR-A07-138 (code, Info) — export_evidence_bundle ZIP is not byte-reproducible (unsorted os.walk, per-file mtimes)
severity Info · reachabilityunreachable· intentintended-and-acceptable· confidence likely · independently confirmed · not reproduced (code trace)
Failure scenario. An auditor compares bundle hashes across two exports and sees different digests for identical content.
Where.forgelm/compliance.py:2315-2319
Recommended fix. Sort dirs/files and use a fixed ZipInfo date_time if a bundle digest is ever published. -
CR-A07-139 (code, Info) — Windows/non-POSIX degradations (flock, SIGALRM ReDoS budget) are silent at runtime, and per-OS tests run only at tag time
severity Info · reachabilitydocumented-opt-in· intentintended-and-acceptable· confidence likely · independently confirmed · not reproduced (code trace)
Failure scenario. A Windows operator running an untrusted-looking regex gets no budget and no warning; a Windows-only regression is found only after tagging.
Where.forgelm/compliance.py:140-157,forgelm/cli/subcommands/_approve.py:28-45,forgelm/_strip_pattern.py:383-386,.github/workflows/publish.yml:44-58
Recommended fix. Log a one-time warning when the budget cannot be armed; add a PR-time Windows smoke job. -
CR-A07-140 (code, Info) — Example 4 emits ForgeLM-reserved event names with payloads that differ from ForgeLM's own
severity Info · reachabilitydocumented-opt-in· intentintended-but-harmful· confidence likely · single reviewer · not reproduced (code trace)
Failure scenario. If a user logger shares an output_dir or run_id with a pipeline run, a user-emitted pipeline.completed may be counted as the pipeline terminal event.
Where.docs/reference/library_api_reference.md:194-207,docs/reference/library_api_reference-tr.md:194-207,docs/reference/audit_event_catalog.md:31,docs/reference/audit_event_catalog.md:37
Recommended fix. Use a namespaced custom event name in the example, and state which names ForgeLM owns. -
CR-A07-141 (code, Info) — A dangling symlink inside the model directory aborts generate_model_integrity, and the trainer swallows it so no model_integrity.json is written
severity Info · reachabilityundocumented· intentintended-but-harmful· confidence confirmed · independently confirmed · reproduced end to end
Failure scenario. final_model contains a broken in-tree symlink (for example a leftover link to a deleted checkpoint file). The integrity manifest is never written, the run logs one WARNING and reports success, and verify-integrity later exits 1 for a missing manifest.
Where.forgelm/compliance.py:787-811,forgelm/trainer.py:2169-2175
Recommended fix. Treat a non-regular or unreadable target like an escaping symlink: record it under skipped_symlinks, or fail the run loudly. -
CR-A07-142 (code, Info) — AuditLogger.log_event (stable API, documented OSError-only) leaks ValueError/TypeError/RecursionError/RuntimeError from its serialisation step for payloads the sanitiser does not normalise; the pipel…
severity Info · reachabilityundocumented· intentcontract-violation· confidence confirmed · independently confirmed · reproduced end to end
Failure scenario. An integrator follows the library guide and logsaudit.log_event('eval.done', per_threshold={0.5: acc, float('nan'): x})or a dict keyed by tuples. They get ValueError/TypeError where the contract promised OSError, so a handler written for OSError lets it crash the job. Inside ForgeLM's pipeline emitter the same payload is dropped from the Art. 12 log with only a WARNING.
Where.forgelm/compliance.py:574-582,forgelm/compliance.py:600-606,forgelm/_strict_json.py:53-54,forgelm/_strict_json.py:78-81
Recommended fix. Either sanitise keys too (non-finite float key -> repr string; reject non-str/int/float/bool/None keys with a clear ValueError), add a cycle and depth guard, and catch Exception around float() in the sanitiser, or document ValueError/TypeError in the log_event contract and the library reference. Do not swallow in _audit_event without an alternate record. -
CR-A07-143 (code, Info) — Writer and verifier HMAC canonicalisations disagree when payload keys collide after JSON key coercion: an untouched keyed log fails verify-audit with exit 6
severity Info · reachabilityundocumented· intentcontract-violation· confidence confirmed · independently confirmed · reproduced end to end
Failure scenario. An orchestrator logs a mapping keyed by both ints and their string forms, for example merged per-epoch metrics from two sources. The nextforgelm verify-audit --require-hmacin CI exits 6, the documented tampering alarm, on an untampered log.
Where.forgelm/compliance.py:574-582,forgelm/compliance.py:2433-2438
Recommended fix. Compute the HMAC over the round-tripped structure (json.loads(dumps_strict(entry, default=str))re-dumped), or HMAC the exact line bytes minus the tag. Alternatively, reject payloads whose keys collide after stringification. -
CR-A07-144 (code, Info) — In-memory verify_pipeline_manifest raises AttributeError on a non-dict root instead of returning a violation, unlike its own handling of non-dict rows
severity Info · reachabilitydefault· intentintended-but-harmful· confidence confirmed · independently confirmed · reproduced end to end
Failure scenario. A library integrator runs verify_pipeline_manifest(json.load(f)) on an untrusted file whose root is an array and gets an AttributeError traceback instead of a violation list.
Where.forgelm/compliance.py:2082,forgelm/compliance.py:3083-3096,forgelm/compliance.py:2097-2109
Recommended fix. Return ['manifest root is , expected a JSON object'] for a non-dict root, mirroring the per-row guard. -
CR-A07-145 (code, Info) — The List[str] facade keeps the routing prefixes, but its docstring omits the integrity-first precedence, and the only implementation of that precedence is private to the CLI
severity Info · reachabilityundocumented· intentintended-but-harmful· confidence likely · independently confirmed · reproduced end to end
Failure scenario. A library CI wrapper maps IO_ERROR:: to 'retry' (exit 2) before checking for untagged violations. A run with one unreadable stage artefact plus a tampered chain is then retried as transient instead of alarming on tampering.
Where.forgelm/compliance.py:3529-3551,forgelm/cli/subcommands/_verify_annex_iv.py:66-133,forgelm/verify/__init__.py:146-157
Recommended fix. State the integrity-first precedence in the facade docstring, or move_classify_pipeline_violations(or a publicpipeline_exit_code(violations)) into forgelm.verify, so library callers share the CLI's routing. -
CR-A07-146 (code, Info) — Stale docstrings describe the pipeline verifier as a per-stage 'training_manifest existence' check reached through verify_pipeline_manifest_at_path
severity Info · reachabilitydefault· intentcontract-violation· confidence likely · single reviewer · not reproduced (code trace)
Failure scenario. A maintainer reasons from these docstrings that the disk layer is an existence check and misjudges what a change affects.
Where.forgelm/compliance.py:3090-3094,forgelm/compliance.py:2063,forgelm/cli/subcommands/_verify_annex_iv.py:139-141,forgelm/cli/subcommands/_verify_annex_iv.py:326-331
Recommended fix. Update the three docstrings to describe the deep parse and Tier-3 corroboration, and say that check 4 runs over well-formed rows. -
CR-A07-147 (code, Info) — Operator-resolution rule names a 'config' source that does not exist and a fallback that differs from the code
severity Info · reachabilitydefault· intentcontract-violation· confidence likely · independently confirmed · reproduced end to end
Failure scenario. An operator follows the standard and looks for a config key to pin the identity; none exists. In a future change, a log_event caller passing a details dict containing 'operator' would make the entry carry that string, for example 'unknown'.
Where.docs/standards/logging-observability.md:108,forgelm/compliance.py:192-229,forgelm/compliance.py:554-561,forgelm/trainer.py:1550
Recommended fix. Fix the standard's wording (env, then user@host, then ConfigError or opt-in anonymous@host). Optionally reject or ignore reserved keys (timestamp, run_id, operator, event, prev_hash, _hmac) in log_event details. -
CR-A07-148 (code, Info) — 'Segregation-of-duties violations are detectable post-hoc' does not hold against the trainer principal, which holds the shared HMAC key
severity Info · reachabilitydefault· intentintended-and-acceptable· confidence likely · single reviewer · reproduced end to end
Failure scenario. A compromised trainer CI approves its own run under a forged approver id. The post-hoc jq SoD cookbook sees two different identities and reports compliance.
Where.forgelm/compliance.py:192-194,forgelm/compliance.py:258,forgelm/cli/subcommands/_approve.py:121-170,docs/guides/human_approval_gate.md:146
Recommended fix. State in the guide that detection assumes approver and trainer use different signing keys. Consider a per-principal key or an asymmetric signature for approval events, and warn when approver == the trainer's operator.
Done when
- Every finding above is fixed or closed with a reason (tick it here).
- Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.
Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsTodo
Activity