[Wave 3] Medium — Module sweep: Config schema & public library API (part 2/2) #546

Wave 3 · 11 finding(s) in 10 fix package(s). Fix these whenever a pull request touches this module ("clean the module you touch").

Findings

• CR-A01-058 (code, Medium) — LoRA user manual documents target_modules: "all-linear", which the List[str] schema rejects; the list form passes --dry-run and fails at model load
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An operator copies the manual's 'all-linear' line: exit 1. They wrap it in a list, pass --dry-run, start a run, and get exit 2 after the model has loaded.
  Where. docs/usermanuals/en/training/lora.md:95-101, docs/usermanuals/tr/training/lora.md:93-99, docs/usermanuals/en/training/lora.md:102, forgelm/config.py:382-385 (confirmed)
  Recommended fix. Type target_modules as Union[Literal['all-linear'], List[str]] and pass the string through to PEFT, or reject ['all-linear'] at config time with a hint. Fix the manual's default claim (EN+TR).

• CR-A01-059 (code, Medium) — User manual misstates the LoRA defaults: target_modules 'Q/K/V/O' (actual default q_proj, v_proj) and rank 16 (actual 8)
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An operator omits target_modules expecting Q/K/V/O, as the manual says, and trains half the documented adapter surface. They also rely on r=16 and get r=8.
  Where. docs/usermanuals/en/training/lora.md:104, docs/usermanuals/tr/training/lora.md:102, docs/usermanuals/en/reference/configuration.md:3, docs/usermanuals/en/reference/configuration.md:56-65
  Recommended fix. Correct both lines in EN and TR to ["q_proj", "v_proj"] and r=8, or change the schema defaults to match the documentation. Annotate the configuration-reference example values as examples, not defaults.

• CR-A01-061 (code, Medium) — Chat user manual (EN+TR) says 'For exact reproducibility, set seed: in YAML'; no seed key exists (config load rejects it, exit 1) and forgelm chat reads no YAML
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end · already tracked: KNOWN-OPEN C31-DOC-SCHEMA-PROSE — docs/roadmap/phase-16-trust-surface-hardening.md:785-786 (S15 open)
  Failure scenario. An operator follows the pitfall advice and adds seed: 42 (top level or under training:). The next forgelm --config run fails at load with 'Extra inputs are not permitted' (exit 1). No setting exists anywhere that makes chat or training seeded.
  Where. docs/usermanuals/en/deployment/chat.md:125, docs/usermanuals/tr/deployment/chat.md:126, forgelm/config.py:1605, forgelm/config.py:449
  Recommended fix. Remove the seed: advice, or implement training.seed (see the LoRA-init finding) and a chat --seed flag. Also correct the /temperature 0 premise.

• CR-A01-063 (code, Medium) — model-merging.md config examples are not runnable configs: merge-only and eval-only YAMLs fail validation, and --benchmark-only ignores the model.name_or_path the page says to set
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An operator copies the TIES example into configs/merge.yaml and runs the documented command. It exits 1 with four missing-field errors and no hint that --merge needs placeholder training and data blocks. The eval step fails the same way.
  Where. docs/usermanuals/en/deployment/model-merging.md:57-68, docs/usermanuals/en/deployment/model-merging.md:83-93, docs/usermanuals/en/deployment/model-merging.md:31-50, docs/usermanuals/tr/deployment/model-merging.md:31-50
  Recommended fix. Either show complete minimal configs with a note that lora, training and data are required but unused by --merge, or relax validation for --merge. Fix line 83 to say the model comes from the --benchmark-only argument. Drop or qualify '(DARE-TIES)'.

• CR-A01-065 (code, Medium) — Auto-chained pipeline stages load the Hub base model through the adapter's base_model_name_or_path with no pin when model.revision is unset; the commit stage 1 was pinned to is not carried forward an…
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. The documented pipeline example (docs/guides/pipeline.md:39-83, root model meta-llama/... with no revision) runs SFT. Upstream pushes a new main commit before the DPO stage starts. The DPO stage silently loads the new base under the SFT adapter: unpinned, unwarned and unrecorded. The resulting model and its Annex IV bundle do not reveal the base mismatch.
  Where. forgelm/config.py:2226-2229, forgelm/model.py:153-160, forgelm/model.py:564-589, forgelm/compliance.py:1296-1298
  Recommended fix. Carry stage 1's resolved base commit forward: - Set LoraConfig.revision (or write the resolved SHA into adapter_config) so the adapter records the base commit. - When the chained input is an adapter directory, resolve and pin its base_model_name_or_path through prepare_revision_pin and record it under role base_model. Also reword the local-path warning so it does not tell operators of adapter cha…

• CR-A01-066 (code, Medium) — LoRA user manual (lora.md EN/TR) misdocuments target_modules and r against the schema: 'all-linear' string rejected, defaults stated as Q/K/V/O and r=16
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An operator copies the documented line lora:\n target_modules: "all-linear" into a valid config. python3 -m forgelm --config B_all_linear.yaml --dry-run exits 1 with 'Configuration validation failed: 1 validation error for ForgeConfig lora.target_modules Input should be a valid list [type=list_type, input_value='all-linear', input_type=str]'. The operator then 'fixes' it to ["all-linear"]. --dry-run exits 0 ('DRY RUN COMPLETE — config is valid'), and --fit-check exits 0 with lora_adapter_…
  Where. docs/usermanuals/en/training/lora.md:97 (confirmed), docs/usermanuals/tr/training/lora.md:84 (not :82), forgelm/config.py:382-385, forgelm/model.py:535
  Recommended fix. Support the PEFT shortcut explicitly: accept the string 'all-linear' (and pass it through as a str in model.py:535 and in the unsloth call at model.py:361), or delete the line from both lora.md pages. In either case, add a validator that rejects a list containing 'all-linear' so the failure moves from exit 2 after the load to exit 1 at --dry-run.

• CR-A01-068 (code, Medium) — No way to select a Hub dataset config (subset) or split: multi-config corpora fail with a remedy ForgeLM rejects, and non-'train' split names (the documented ultrachat_200k example) crash with KeyErr…
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An operator follows data_preparation.md and sets dataset_name_or_path: HuggingFaceH4/ultrachat_200k, or picks a common multi-config corpus such as one with main/socratic subsets. The run downloads and loads the base model, then fails with exit 2 and either "'train'" or an instruction to call load_dataset(..., '') that no YAML key can express. The only workaround is to pre-export the subset to local JSONL, which also drops the Hub revision record ('local_path').
  Where. forgelm/config.py:804-826 (DataConfig, extra='forbid', no name/split field), forgelm/data.py:297-337 (_load_single_dataset; load_dataset calls at 322, 332, 337 pass no name/split), forgelm/data.py:532 (_merge_extra_datasets also indexes primary_dataset['train']), forgelm/data.py:565
  Recommended fix. Add data.dataset_config_name and data.split (or a split mapping) fields that are forwarded to load_dataset, and add data.revision (see the reproducibility finding). Make _ensure_validation_split fail with an actionable error that lists the available split names. Fix data_preparation.md's example or document the limitation.

• CR-A01-069 (code, Medium) — The training corpus is the one Hub input with no revision knob: every rerun of the same YAML reads the current default-branch head, while README, configuration.md, safety_compliance.md and the run lo…
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · not reproduced (code trace)
  Failure scenario. A team reruns an approved YAML weeks later to reproduce a model, or to retrain after an erasure request. The upstream Hub dataset has received commits, so the rerun trains on different rows. The log says 'pinned to Hub revision ', the config_hash matches, and nothing warns. Only a diff of data_provenance.json hf_revision between the two bundles reveals it, and an auditor relying on safety_compliance.md believes the ID plus timestamp proves 'the exact same data was used'.
  Where. forgelm/config.py:804-826, forgelm/data.py:223-283 (dataset_info call at 265; DEBUG failure logs at 267, 272), forgelm/data.py:275, forgelm/data.py:325-337
  Recommended fix. Add an optional data.revision (and a per-extra revision) that is validated like model.revision and passed to load_dataset. Emit a WARNING when a Hub dataset is loaded without an operator pin, or when the lookup fails. Reword the INFO line to 'resolved and pinned for this run'. Update configuration.md, README:121 and safety_compliance.md:376-386 to state the dataset gap and the hf_revision grades.

• CR-A01-070 (code, Medium) — No path-alias check: synthetic.output_file may equal seed_file or data.dataset_name_or_path, so --generate-data silently rewrites the seed set or the curated training set (exit 0), and later runs reg…
  severity Medium · reachability documented-opt-in · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An operator who wants a 'replay' file sets output_file to the same JSONL as seed_file. The docs never say that teacher_backend: file reads its responses from seed_file. The first run destroys the seed set and exits 0. Every later api-backend run pays the teacher to answer the previous run's JSON lines and still exits 0. A different operator points output_file at the curated dataset and loses it on the first run, with no warning.
  Where. forgelm/config.py:1356-1363, forgelm/config.py:1420-1442, forgelm/config.py:807-809, forgelm/synthetic.py:104-105
  Recommended fix. At config validation, and again at the start of generate(), refuse an output_file that resolves to the same file (os.path.realpath, or os.path.samefile when both exist) as synthetic.seed_file or the config file. When it resolves to data.dataset_name_or_path or one of data.extra_datasets and already exists, warn or require an explicit opt-in such as synthetic.overwrite: true or --force. Do not…

• CR-A01-075 (code, Low) — No audit event for config validation pass/fail, which the logging standard makes mandatory; the strict risk-tier refusal leaves no Art. 12 trace
  severity Low · reachability default · intent contract-violation · confidence likely · single reviewer · reproduced (plumbing only)
  Failure scenario. An auditor reconstructing why a high-risk training job never ran finds no record in any audit log: the refusal exists only in the transient CI stderr.
  Where. forgelm/cli/_config_load.py:17-47, forgelm/config.py:1837-1842, docs/standards/logging-observability.md:110-113, docs/reference/audit_event_catalog.md
  Recommended fix. Either emit a config.validation_failed / config.validated event once training.output_dir is resolvable (and catalogue it), or amend logging-observability.md to state that config-time refusals are surfaced by exit 1 plus the JSON envelope rather than by the audit log.

• ↳ same root cause: DOC-D1B07-014 (docs, Medium) — docs/standards/logging-observability.md: false claim — Audit events MUST be emitted for 'Config validation pass/fail'
  severity Medium
  Where. docs/standards/logging-observability.md:113 · type: false claim
  The document says. Audit events MUST be emitted for 'Config validation pass/fail'
  What the code does. No config-validation audit event exists in code or in the event catalog. A config error exits 1 from _load_config_or_exit before any AuditLogger exists.
  Code references. forgelm/cli/_config_load.py:17-46, docs/reference/audit_event_catalog.md:30-110
  Proposed correction. Either implement and catalogue an event (for example config.validated) or remove the line, stating that config failures are reported by exit code 1 and stderr only.

Done when

• Every finding above is fixed or closed with a reason (tick it here).
• Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}