Repository navigation
[Wave 3] Medium — Security-contract fixes: Exit-code and error/output contract (part 1/2) #79
Description
Wave 3 · 30 finding(s) in 20 fix package(s). Medium-severity findings that touch the security contract (exit codes, secrets, paths, provenance): do these before the rest of wave 3.
Findings
-
CR-A01-029 (code, Medium) — Documented ConfigError contract is false for ForgeConfig(**dict): most failures raise pydantic ValidationError, the strict-tier refusal raises ConfigError
severity Medium · reachabilitydefault· intentcontract-violation· confidence confirmed · independently confirmed · reproduced end to end
Failure scenario. A parametric sweep script wraps ForgeConfig(**params) inexcept ConfigError, as the reference doc says. A sweep point with a bad learning rate raises ValidationError and the uncaught exception kills the whole sweep. A script that usesexcept ValidationErrorinstead lets the high-risk refusal escape as an unexpected exception.
Where.forgelm/config.py:1836-1842,forgelm/config.py:1956-1967,forgelm/config.py:1970-1973,docs/reference/library_api_reference.md:48(line 46, load_config, is correct)
Recommended fix. Either make ConfigError subclass ValueError (Pydantic then wraps it, and load_config still rewraps it as ConfigError), or document the dual contract in library_api_reference.md:48 (+TR): 'ForgeConfig(**d) raises pydantic.ValidationError for schema errors and ConfigError for the strict risk-tier gate; load_config normalises both to ConfigError'. -
CR-A01-032 (code, Medium) — trl's GRPO rule that
grpo_num_generationsmust divide the generation batch is not validated; the documented example passes--dry-runand then exits 2
severity Medium · reachabilitydefault· intentcontract-violation· confidence confirmed · independently confirmed · reproduced (plumbing only)
Failure scenario. An operator copies the manual's GRPO example. The dry-run says the config is valid, and the real run exits 2 after the model loads.
Where.forgelm/config.py:510-512,forgelm/trainer.py:758,docs/usermanuals/en/training/grpo.md:53
Recommended fix. Add a ForgeConfig model_validator for GRPO divisibility (single-process view) with an actionable message, and fix the manual's example. -
CR-A01-035 (code, Medium) — report_to wandb/mlflow with the tracking extra missing passes --dry-run, then crashes in transformers with exit 2 and no forgelm[tracking] hint
severity Medium · reachabilitydefault· intentcontract-violation· confidence confirmed · independently confirmed · reproduced end to end
Failure scenario. In CI, a config with report_to: mlflow passes validation, spends GPU time loading the model, then exits 2. The pipeline treats that as a retryable infra crash rather than a config/install fix, and the hint names a bare package instead of the supported extra.
Where.forgelm/config.py:604-607,forgelm/trainer.py:519(confirmed),forgelm/cli/_training.py:127-133(disputed),docs/standards/error-handling.md:67
Recommended fix. Probe importlib.util.find_spec('wandb'/'mlflow') in a ForgeConfig validator or at dry-run/trainer setup, and raise ImportError (mapped to exit 1) naming the forgelm[tracking] / forgelm[tracking-mlflow] extra. -
CR-A01-037 (code, Medium) — webhook.tls_ca_bundle is not validated; a wrong path raises OSError out of notify_start and aborts training with exit 2 (dry-run passes)
severity Medium · reachabilitydocumented-opt-in· intentcontract-violation· confidence confirmed · independently confirmed · reproduced end to end
Failure scenario. A typo in the corporate CA bundle path kills every training run at start with a runtime-error exit code and an incomplete audit trail, while dry-run validation stays green.
Where.forgelm/config.py:1501-1504,forgelm/webhook.py:132,forgelm/webhook.py:156-229,forgelm/trainer.py:1676
Recommended fix. Validate path existence at config load (exit 1) and catch OSError in _post_payload as a best-effort transport failure. -
CR-A01-067 (code, Medium) — Config-shaped LoRA errors (empty or unmatched target_modules, invalid task_type) pass validation and --dry-run, then fail after the full model load with exit 2 instead of 1
severity Medium · reachabilitydefault· intentcontract-violation· confidence confirmed · independently confirmed · reproduced end to end
Failure scenario. cfg D_empty.yaml (lora: {target_modules: []}): --dry-run exits 0 ('config is valid'), --fit-check exits 0 (adapter 0.0 GB). The real run logs 'Loading Base Model ...', 'Setting up PEFT configuration', then '[ERROR] forgelm.cli: Training pipeline failed.' with the PEFT ValueError traceback, and exits 2. On a 70B or Hub-downloaded model the failure comes only after the whole download and load. CI routing that treats exit 1 as 'fix your config' and exit 2 as 'infrastructure/training failure' mis…
Where.forgelm/config.py:386-388(task_type: str),forgelm/model.py:609(get_peft_model raises for empty or unmatched target_modules),forgelm/cli/_training.py:141,forgelm/cli/_training.py:173-180(broad except -> EXIT_TRAINING_ERROR)
Recommended fix. Add Field(min_length=1) and reject empty strings in the list. Wrap the get_peft_model ValueError for 'not found' / 'Notarget_modules' in a ConfigError, so the CLI exits 1 and names the model's actual Linear leaf names. Optionally, resolve target_modules against model.named_modules() before get_peft_model, which would also cover M5r2-A03-002. -
CR-A02-049 (code, Medium) — Code does not match forgelm/config.py: final_model_dir: Subdirectory of output_dir where the final promoted model lands.
severity Medium · reachabilitydocumented-opt-in· intentcontract-violation· confidence confirmed · independently confirmed
Failure scenario. The document states: final_model_dir: Subdirectory of output_dir where the final promoted model lands.. Observed in the code: The trainer honours it (trainer.py:1590-1593). forgelm purge hardcodes 'final_model': the memorisation warning (cli/subcommands/_purge.py:401), staging targets (:970) and the retention scan (:1282-1287). With a custom final_model_dir, the erasure-warning event is not emitted and per-run staging dirs are neither found nor purged.
Where.forgelm/trainer.py:1590-1593,forgelm/cli/subcommands/_purge.py:401-404,forgelm/cli/subcommands/_purge.py:970-973,forgelm/cli/subcommands/_purge.py:1282-1287 -
↳ same root cause: DOC-BFIELD-008 (docs, Medium) — forgelm/config.py: misleading — final_model_dir: Subdirectory of output_dir where the final promoted model lands.
severity Medium
Where.forgelm/config.py:454-456· type: misleading
The document says. final_model_dir: Subdirectory of output_dir where the final promoted model lands.
What the code does. The trainer honours it (trainer.py:1590-1593). forgelm purge hardcodes 'final_model': the memorisation warning (cli/subcommands/_purge.py:401), staging targets (:970) and the retention scan (:1282-1287). With a custom final_model_dir, the erasure-warning event is not emitted and per-run staging dirs are neither found nor purged.
Code references.forgelm/trainer.py:1590-1593,forgelm/cli/subcommands/_purge.py:401,forgelm/cli/subcommands/_purge.py:970,forgelm/cli/subcommands/_purge.py:1282-1287 -
CR-A02-057 (code, Medium) — TrainResult.error keeps only the last gate's reason: earlier kept failures are overwritten, a kept loss-gate breach is never recorded, and the judge has no pass flag in the envelope
severity Medium · reachabilitydefault· intentintended-but-harmful· confidence confirmed · independently confirmed · reproduced with stubs · already tracked: PARTIALLY-FIXED C33-TRAIN-JSON-CONTRACT — docs/roadmap/phase-16-trust-surface-hardening.md:291,308 (S2 done); CHANGELOG.md:102-105
Failure scenario. A library caller or dashboard reads TrainResult.error after a run in which the benchmark and judge failed while the model was kept. It sees only the judge reason and concludes that the benchmark passed. A diverged (NaN) model kept under auto_revert=false returns error=None. With auto_revert=true, a reverted run's envelope blames only the judge, hiding that the safety classifier never answered, which is the operator-actionable infrastructure fault.
Where.forgelm/trainer.py:1351,forgelm/trainer.py:1372-1373,forgelm/trainer.py:1458-1460,forgelm/trainer.py:898-900
Recommended fix. Append reasons (for example '; '.join, or a gate_failures list on TrainResult) instead of assigning them. Record kept loss-gate breaches through the same helper. Add judge_passed to TrainResult, the envelope and the manifest. Add a test with several failing gates. -
CR-A03-021 (code, Medium) — Deterministic dataset/schema errors surface after the model load as exit 2, with a repr-mangled message (also in the JSON envelope)
severity Medium · reachabilitydefault· intentcontract-violation· confidence confirmed · independently confirmed · reproduced end to end
Failure scenario. CI retry logic keyed on exit 2 re-runs a deterministic failure; the operator reading the JSON error sees escaped garbage.
Where.forgelm/data.py:593-615,forgelm/data.py:656-660,forgelm/cli/_training.py:141-142,forgelm/cli/_training.py:173-180
Recommended fix. Raise ConfigError (or ValueError with a message) instead of KeyError so str(e) is clean and the handler maps it to exit 1; validate dataset schema/path in --dry-run or before the model load. -
CR-A03-028 (code, Medium) — Alpaca-style
inputcolumn is silently dropped from instruction/output rows
severity Medium · reachabilitydefault· intentintended-but-harmful· confidence confirmed · independently confirmed · reproduced end to end
Failure scenario. A summarisation or extraction fine-tune learns to hallucinate answers with no source text.
Where.forgelm/data.py:452-453,forgelm/data.py:481-488,forgelm/data.py:684
Recommended fix. Includeinput(appended to the user turn) or reject unknown extra columns loudly. -
CR-A03-032 (code, Medium) — --fit-check output contract: ad-hoc JSON without success or notes, and exit 0 whatever the verdict (OOM/TIGHT included)
severity Medium · reachabilitydefault· intentcontract-violation· confidence confirmed · independently confirmed · reproduced end to end
Failure scenario. A pipeline written against the documented universalsuccesskey treats a missing key as failure or crashes.
Where.forgelm/cli/_fit_check.py:16-30,docs/usermanuals/en/reference/json-output.md:15(confirmed),tools/check_strict_json_writers.py:50,forgelm/cli/_fit_check.py:17-29
Recommended fix. Add success: true (keep existing keys), document the envelope, and use dumps_strict. -
↳ same root cause: DOC-D3B02-030 (docs, Medium) — docs/usermanuals/en/operations/cicd.md: misleading — --fit-check as a CI/CD step with JSON output; TIGHT is a hard refusal
severity Medium
Where.docs/usermanuals/en/operations/cicd.md:58-60; docs/reference/usage.md:128-133 · type: misleading
The document says. --fit-check as a CI/CD step with JSON output; TIGHT is a hard refusal
What the code does. Exit 0 for every verdict; the step cannot fail on OOM.
Code references.forgelm/cli/_no_train_modes.py:269-271
Proposed correction. Document the exit code or add a gating option (F8).
Turkish mirror. the Turkish page has the same error; fix both in the same PR -
↳ same root cause: DOC-D3B03-010 (docs, Medium) — docs/usermanuals/en/reference/json-output.md: false claim — Every envelope starts with "success": true | false.
severity Medium
Where.docs/usermanuals/en/reference/json-output.md:15(Turkish mirror:docs/usermanuals/tr/reference/json-output.md:15) · type: false claim
The document says. Every envelope starts with "success": true | false.
What the code does.forgelm --config X --fit-check --output-format jsonemits {verdict, estimated_gb, available_gb, hypothetical, breakdown, recommendations} with no success key (I ran it). A pipeline run's JSON summary is asdict(PipelineState) {pipeline_run_id, ..., final_status, stages}, also without a success key (I ran a two-stage pipeline).
Code references.forgelm/cli/_fit_check.py:15-29,forgelm/cli/_pipeline.py:282-284,forgelm/cli/_pipeline.py:1669-1672
Proposed correction. Fix the code to add success to the fit-check and pipeline-summary envelopes, and document both envelopes on this page.
Turkish mirror. the Turkish page has the same error; fix both in the same PR -
CR-A03-033 (code, Medium) — architecture.md claims no HTTP-discipline carve-outs, but Hub/datasets/lm-eval SDK traffic bypasses _http.py unmarked
severity Medium · reachabilitydefault· intentcontract-violation· confidence confirmed · independently confirmed · reproduced end to end
Failure scenario. The document states: 'every outbound HTTP call from forgelm/ goes through forgelm/_http.py' ... 'No HTTP-discipline carve-outs exist today; if one is ever needed (e.g., a cloud provider's SDK that wraps its own HTTP), document it inline + add a # noqa marker.'. Observed in the code: Hugging Face Hub and datasets traffic goes out through third-party SDK HTTP with no _http.py policy and no marker. Examples: HfApi().dataset_info, load_dataset(path) (including a URL or data_files path), and every f…
Where.forgelm/data.py:261-273,forgelm/data.py:332,forgelm/judge.py:377-378,forgelm/trainer.py:1167-1170
Recommended fix. Scope the rule: 'every direct HTTP call ForgeLM makes (webhook, judge, synthetic teacher, doctor probe) goes through _http.py; Hugging Face Hub, datasets and lm-eval downloads use their own HTTP stacks, are governed by HF_ENDPOINT/offline flags, and are not covered by the SSRF guard.' Add the documented markers at those sites. -
↳ same root cause: DOC-D1B07-007 (docs, Medium) — docs/standards/architecture.md: misleading — 'every outbound HTTP call from forgelm/ goes through forgelm/_http.py' ... 'No HTTP-discipline carve-outs exist today; if one is ever needed (e.g., a clou…
severity Medium
Where.docs/standards/architecture.md:276,286,312 · type: misleading
The document says. 'every outbound HTTP call from forgelm/ goes through forgelm/_http.py' ... 'No HTTP-discipline carve-outs exist today; if one is ever needed (e.g., a cloud provider's SDK that wraps its own HTTP), document it inline + add a # noqa marker.'
What the code does. Hugging Face Hub and datasets traffic goes out through third-party SDK HTTP with no _http.py policy and no marker. Examples: HfApi().dataset_info, load_dataset(path) (including a URL or data_files path), and every from_pretrained. The SSRF guard (private/loopback/IMDS refusal, redirect refusal) does not apply to these calls.
Code references.forgelm/data.py:273,forgelm/data.py:322-337,forgelm/judge.py:378-379,forgelm/trainer.py:1155-1168
Proposed correction. Scope the rule: 'every direct HTTP call ForgeLM makes (webhook, judge, synthetic teacher, doctor probe) goes through _http.py; Hugging Face Hub, datasets and lm-eval downloads use their own HTTP stacks, are governed by HF_ENDPOINT/offline flags, and are not covered by the SSRF guard.' Add the documented markers at those sites. -
CR-A03-034 (code, Medium) — Documented optional preference-row 'system' (and prompt_messages) fields are silently ignored
severity Medium · reachabilitydefault· intentcontract-violation· confidence confirmed · independently confirmed · reproduced (plumbing only)
Failure scenario. The document states: Optional 'system' string column is merged into the prompt at training time / used for both responses. Observed in the code: Preference datasets pass through unchanged. TRL's DPOTrainer ignores a plain 'system' column, and the probe shows the prompt without the system text.
Where.forgelm/data.py:634-645(_passthrough_for_trainer; preference branch 636-638),forgelm/data.py:634-645(confirmed),docs/usermanuals/en/concepts/data-formats.md:65-67(TR mirror docs/usermanuals/tr/concepts/data-formats.md:67),forgelm/data.py:724-726
Recommended fix. Merge the system column into the prompt in data.py, or remove the claim -
↳ same root cause: DOC-D3B05-046 (docs, Low) — docs/usermanuals/en/training/dpo.md: false claim — Optional 'system' string column is merged into the prompt at training time / used for both responses
severity Low
Where.docs/usermanuals/en/training/dpo.md:101; docs/usermanuals/en/concepts/data-formats.md:65-66 · type: false claim
The document says. Optional 'system' string column is merged into the prompt at training time / used for both responses
What the code does. Preference datasets pass through unchanged. TRL's DPOTrainer ignores a plain 'system' column, and the probe shows the prompt without the system text.
Code references.forgelm/data.py:634-638
Proposed correction. Merge the system column into the prompt in data.py, or remove the claim
Turkish mirror. the Turkish page has no such statement -
CR-A04-016 (code, Medium) — Single-stage --resume (and bare --force-resume) is silently ignored on pipeline configs
severity Medium · reachabilitydefault· intentcontract-violation· confidence confirmed · independently confirmed · reproduced end to end
Failure scenario. A preemptible CI job uses the documented --resume on a pipeline config. Every restart silently re-trains the whole chain from scratch and replaces completed artefacts.
Where.forgelm/cli/_pipeline.py:1725-1754,forgelm/cli/_pipeline.py:1704-1716,forgelm/cli/_dispatch.py:245-246,forgelm/cli/_pipeline.py:1738-1754(confirmed)
Recommended fix. Add ('resume','--resume') to _UNSUPPORTED_ON_PIPELINE, pointing to --resume-from, and reject --force-resume without --resume-from. -
CR-A04-020 (code, Medium) — Explicit --resume PATH is validated only after the full model load and dataset preparation
severity Medium · reachabilitydefault· intentcontract-violation· confidence confirmed · independently confirmed · reproduced end to end
Failure scenario. A CI retry passes a wrong checkpoint path. It wastes a full model load on a large model and returns no machine-readable error.
Where.forgelm/cli/_training.py:141-146,forgelm/cli/_resume.py:26-30,forgelm/cli/_resume.py:13-19,forgelm/cli/_training.py:140-149(confirmed)
Recommended fix. Resolve or validate the resume checkpoint before loading the model. Have _resume raise ConfigError, which the existing handler turns into an envelope, instead of calling sys.exit. -
CR-A04-026 (code, Medium) — CLI seam has no catch-all: exceptions escaping a dispatcher or no-train mode exit 1 (config-error class) with a traceback and no JSON envelope
severity Medium · reachabilitydefault· intentcontract-violation· confidence confirmed · independently confirmed · reproduced end to end · already tracked: KNOWN-OPEN C33-CLI-ERROR-ENVELOPE — docs/roadmap/phase-16-trust-surface-hardening.md:497-508 (S7 open)
Failure scenario. A disk-full or permission error during --compliance-export exits 1, the 'config error' class, and prints a traceback with empty stdout. CI treats it as an operator typo rather than a runtime failure, and JSON consumers get nothing to parse.
Where.forgelm/cli/_no_train_modes.py:264-283(confirmed),forgelm/cli/_no_train_modes.py:66-68,forgelm/cli/_no_train_modes.py:130-132(confirmed),forgelm/cli/_no_train_modes.py:184-186
Recommended fix. Wrap each no-train mode in the same ConfigError->1 / ImportError->2 / Exception->2 boundary with the JSON envelope used by _run_training_pipeline, and add 'error' to failure envelopes. -
CR-A04-028 (code, Medium) — Failures in model load, dataset preparation or trainer construction leave no audit event and no failure webhook
severity Medium · reachabilitydefault· intentcontract-violation· confidence confirmed · independently confirmed · reproduced end to end
Failure scenario. The document states: pipeline.initialized is emitted before any model load.. Observed in the code: Both the CLI and the pipeline orchestrator load the model and prepare the dataset before ForgeTrainer is constructed. The event is emitted in ForgeTrainer.init, and pipeline.failed only inside ForgeTrainer.train(), so a failed model load or dataset preparation leaves no audit entry at all.
Where.forgelm/cli/_training.py:141-148(confirmed),forgelm/cli/_pipeline.py:1573-1576,forgelm/trainer.py:354-357,forgelm/trainer.py:1693-1709
Recommended fix. Doc: 'Emitted when ForgeTrainer is constructed, after the model and dataset have been loaded. A failure during model loading or dataset preparation is not recorded in audit_log.jsonl.' Code: emit a start event, and pipeline.failed, around the load phase. -
↳ same root cause: DOC-D1B04-018 (docs, Medium) — docs/reference/audit_event_catalog.md: false claim — pipeline.initialized is emitted before any model load.
severity Medium
Where.docs/reference/audit_event_catalog.md:30(Turkish mirror:docs/reference/audit_event_catalog-tr.md:30) · type: false claim
The document says. pipeline.initialized is emitted before any model load.
What the code does. Both the CLI and the pipeline orchestrator load the model and prepare the dataset before ForgeTrainer is constructed. The event is emitted in ForgeTrainer.init, and pipeline.failed only inside ForgeTrainer.train(), so a failed model load or dataset preparation leaves no audit entry at all.
Code references.forgelm/cli/_training.py:141-148,forgelm/cli/_pipeline.py:1564-1566,forgelm/trainer.py:355-357,forgelm/trainer.py:1702
Proposed correction. Doc: 'Emitted when ForgeTrainer is constructed, after the model and dataset have been loaded. A failure during model loading or dataset preparation is not recorded in audit_log.jsonl.' Code: emit a start event, and pipeline.failed, around the load phase.
Turkish mirror. the Turkish page has the same error; fix both in the same PR -
↳ same root cause: DOC-D1B04-025 (docs, Medium) — docs/reference/audit_event_catalog.md: wrong detail — training.start fires when train() is entered, before model load; training.failure covers a dataset error.
severity Medium
Where.docs/reference/audit_event_catalog.md:162,164 (Turkish mirror:docs/reference/audit_event_catalog-tr.md:162,164) · type: wrong detail
The document says. training.start fires when train() is entered, before model load; training.failure covers a dataset error.
What the code does. The model is loaded and the dataset prepared before train() (and before ForgeTrainer exists). A failure there goes to _report_training_error: no training.failure webhook and no pipeline.failed event.
Code references.forgelm/cli/_training.py:141-148,forgelm/cli/_training.py:160-178,forgelm/trainer.py:1677-1712
Proposed correction. 'training.start fires when train() is entered (after model and dataset loading). Errors while loading the model or preparing the dataset produce no webhook.' Code: notify failure for the load phase.
Turkish mirror. the Turkish page has the same error; fix both in the same PR -
↳ same root cause: DOC-D1B04-046 (docs, Medium) — docs/reference/webhook_schema.md: wrong detail — training.start fires 'before model load'; training.failure covers a 'dataset error'.
severity Medium
Where.docs/reference/webhook_schema.md:35,37 (Turkish mirror:docs/reference/webhook_schema-tr.md:35,37) · type: wrong detail
The document says. training.start fires 'before model load'; training.failure covers a 'dataset error'.
What the code does. The model and dataset are loaded before train() is called. Errors in model loading or dataset preparation produce no webhook at all.
Code references.forgelm/cli/_training.py:141-148,forgelm/cli/_training.py:160-178,forgelm/trainer.py:1677
Proposed correction. 'training.start: train() entered (after model and dataset loading). Failures while loading the model or preparing the dataset emit no webhook.'
Turkish mirror. the Turkish page has the same error; fix both in the same PR -
↳ same root cause: DOC-D1B07-013 (docs, Medium) — docs/standards/logging-observability.md: misleading — Audit events MUST be emitted for training failure; webhook
training.failurefires when 'Training itself crashed (OOM, dataset error, exception…
severity Medium
Where.docs/standards/logging-observability.md:110-113,162 · type: misleading
The document says. Audit events MUST be emitted for training failure; webhooktraining.failurefires when 'Training itself crashed (OOM, dataset error, exception in pipeline)'
What the code does. The model and dataset are loaded in the CLI before ForgeTrainer exists. A failure there (model download, dataset load or format) reaches the CLI's generic catch, which only logs and exits 2: there is no AuditLogger, so nopipeline.failedevent, and no notifier, so notraining.failurewebhook. Only failures inside train() emit both.
Code references.forgelm/cli/_training.py:141-148,forgelm/cli/_training.py:173-180,forgelm/trainer.py:1677-1709
Proposed correction. Fix the code (create the audit logger and notifier before model and dataset load, or emit from the CLI catch), or qualify the doc: 'Failures before the trainer is constructed (model or dataset load) currently emit neither an audit event nor a webhook.' -
CR-A05-012 (code, Medium) — Exit-code clamp promise ('never outside 0-6', signal-derived 128+N clamped) does not hold: SIGTERM exits 143, stdout flush failure exits 120
severity Medium · reachabilitydefault· intentcontract-violation· confidence confirmed · independently confirmed · reproduced end to end
Failure scenario. A pipeline that branches only on the documented 0-6 set receives 143 when its runner times out the job, and falls into an unhandled branch.
Where.forgelm/cli/_exit_codes.py:1-6(docstring: any other value, e.g. signal-derived 128+N, is clamped),forgelm/cli/_exit_codes.py:54-62(_clamp_exit_code applies only to returned values),forgelm/cli/_dispatch.py:113-121(the subcommand try block that catches only KeyboardInterrupt),forgelm/cli/_dispatch.py:210
Recommended fix. Either make the promise true where it can be (a SIGTERM handler that raises KeyboardInterrupt or exits 2, and a flush of stdout inside main() with BrokenPipeError mapped to a public code), or narrow the docstring and docs to 'codes ForgeLM itself returns are 0-6; external signals and broken pipes are outside the contract'. -
CR-A05-016 (code, Medium) — Code does not match docs/guides/gdpr_erasure.md: Exit 1 covers a mutually-exclusive flag combination.
severity Medium · reachabilitydefault· intentcontract-violation· confidence confirmed · independently confirmed
Failure scenario. The document states: Exit 1 covers a mutually-exclusive flag combination.. Observed in the code: --row-id, --run-id and --check-policy form an argparse mutually-exclusive group, and --kind uses argparsechoices. Argparse rejects violations with exit 2. Reproduced:purge --row-id x --run-id ygave EXIT=2,--row-id x --check-policygave EXIT=2,--kind logsgave EXIT=2. Only the no-mode case exits 1.
Where.forgelm/cli/_parser.py:1096-1127,forgelm/cli/_exit_codes.py:10-12,docs/guides/gdpr_erasure.md:112,docs/guides/gdpr_erasure-tr.md:112
Recommended fix. Fix the code so argparse usage errors exit 1, or document: "Flag-combination and invalid--kinderrors are rejected by the argument parser with exit 2." -
↳ same root cause: DOC-D2B02-027 (docs, Medium) — docs/guides/gdpr_erasure.md: wrong detail — Exit 1 covers a mutually-exclusive flag combination.
severity Medium
Where.docs/guides/gdpr_erasure.md:112(Turkish mirror:docs/guides/gdpr_erasure-tr.md:112) · type: wrong detail
The document says. Exit 1 covers a mutually-exclusive flag combination.
What the code does. --row-id, --run-id and --check-policy form an argparse mutually-exclusive group, and --kind uses argparsechoices. Argparse rejects violations with exit 2. Reproduced:purge --row-id x --run-id ygave EXIT=2,--row-id x --check-policygave EXIT=2,--kind logsgave EXIT=2. Only the no-mode case exits 1.
Code references.forgelm/cli/_parser.py:1096-1129,forgelm/cli/_exit_codes.py:10-12
Proposed correction. Fix the code so argparse usage errors exit 1, or document: "Flag-combination and invalid--kinderrors are rejected by the argument parser with exit 2."
Turkish mirror. the Turkish page has the same error; fix both in the same PR -
CR-A07-036 (code, Medium) — verify-integrity (and verify-gguf) read their input without a regular-file check or a guard for non-JSONDecodeError parse errors: FIFO hang, unbounded device read, traceback without an envelope
severity Medium · reachabilitydefault· intentcontract-violation· confidence confirmed · independently confirmed · reproduced end to end · already tracked: KNOWN-OPEN VERIFY-UNCAPPED-READ — docs/roadmap/risks-and-decisions.md:263
Failure scenario. A third-party model bundle ships model_integrity.json as a FIFO, or a symlink to /dev/zero. The CI verify-integrity step hangs until the job times out or exhausts memory, and no verdict or envelope is produced.
Where.forgelm/verify/_model_integrity.py:86,forgelm/cli/subcommands/_verify_integrity.py:85,forgelm/verify/_model_integrity.py:81-86(confirmed),forgelm/verify/_gguf.py:82-83(confirmed)
Recommended fix. Catch ValueError and RecursionError from the manifest parse in the dispatcher (or convert them in verify_integrity) and route them to EXIT_CONFIG_ERROR with the envelope. -
CR-A07-053 (code, Medium) — is_audit_integrity_failure cannot be fed from the public verify_audit_log; passing the result silently returns False
severity Medium · reachabilitydocumented-opt-in· intentcontract-violation· confidence confirmed · independently confirmed · reproduced end to end
Failure scenario. A pipeline calls the public verify_audit_log and then is_audit_integrity_failure(result) to decide between paging security (6) and reporting an operator error (1). A tampered audit log is classified as a non-integrity input error, so the tamper alarm the docs tie to exit 6 never fires. valid stays False, so this is a misclassification, not a pass.
Where.forgelm/verify/_audit_log.py:18-58,forgelm/compliance.py:2770-2819,forgelm/compliance.py:2822,docs/reference/library_api_reference.md:89
Recommended fix. Either make is_audit_integrity_failure accept a VerifyResult (raising TypeError for anything that is not a token or a VerifyResult), or expose the classified verifier publicly. Otherwise state in the doc that the audit predicate needs the private _verify_audit_log_classified token. Also reconcile the forgelm/verify/init.py docstring ('no stability guarantee') with the doc ('stable enough in p…
Done when
- Every finding above is fixed or closed with a reason (tick it here).
- Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.
Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsTodo
Activity