Skip to content

Pipeline-level audit emission swallows every failure, including the AuditLogger re-root refusal, so the chain runs to exit 0 with pipeline.* decision records missing (CR-A04-003) #372

Open

Description

@cemililik

Wave 1 · theme D1-2: Audit chain, integrity manifest and verifier robustness · highest severity High · 1 finding(s) · effort small

Summary

Someone truncates the pipeline audit log. The next run, possibly with --force-resume over an edited config, completes with exit 0 and leaves no Art. 12 record of the run, its forced resume or its stage decisions.

Primary finding

CR-A04-003 — Pipeline-level audit emission swallows every failure, including the AuditLogger re-root refusal, so the chain runs to exit 0 with pipeline.* decision records missing

severity High · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end

What happens. PipelineOrchestrator._audit_event (forgelm/cli/_pipeline.py:1408-1420) wraps both AuditLogger construction and log_event in 'except Exception' and only logs a WARNING ('best-effort telemetry'). This downgrades the ConfigError that AuditLogger raises on purpose (re-root refusal for a truncated or unterminated log, unresolved operator identity) and the OSError from a failed write ('retry or fail the run'). pipeline.force_resume, resume_refused, stage_gated, stage_reverted and completed are dropped while --force-resume still overwrites the stored config hash and the run exits 0. The single-stage trainer and approve/reject treat the same failures as fatal. test_audit_event_failures_do_not_abort_pipeline pins the swallow.

Where.

Evidence.

  • e2e_trunc/: copied pl/audit_log.jsonl.manifest.json without the log, then ran the pipeline. 6x '[ERROR] AUDIT INTEGRITY ... absent or empty' followed by '[WARNING] Failed to emit audit event ... Audit log re-root refused'. Both stages trained; final_status completed; EXIT=0; pl/ has no audit_log.jsonl. verify-annex-iv --pipeline afterwards: 'unattested (audit_log_absent)' exit 1, the same exit as a clean run without FORGELM_AUDIT_SECRET.
  • Setup: chmod 444 on pt_run/audit_log.jsonl, then the YAML edited (hash changed). --resume-from sft_b --force-resume logged a WARNING 'Failed to emit audit event 'pipeline.force_resume': ... The hash chain has NOT been advanced — retry or fail the run.' The run exited 0 and the audit log still had 6 lines. The stored hash had been replaced, so the next resume with the edited YAML succeeded with no recorded override.
  • Probe E3: AuditLogger.log_event patched to raise OSError for pipeline.force_resume. The forced resume exits 0, the audit_log.jsonl has no force_resume event, and only a WARNING is logged.

Recommended fix. Let ConfigError (re-root refusal, operator identity) propagate so the run maps to exit 1, as in single-stage mode. Keep best-effort only for OSError if at all, and record an audit-unavailable marker in pipeline_state.

Acceptance criteria

  • A regression test reproduces the failure scenario and asserts the corrected behaviour.
  • Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
  • Every finding in this issue is resolved or explicitly closed with a reason.

Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: pipelineCode area: pipelinebugSomething isn't workingseverity: highWrong behaviour or a false claim users rely onsource: review-2026-10Found by the October 2026 code and documentation reviewwave: 1Roadmap wave 1: high severity, small effort

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions