[Wave 4] Low/info — Module sweep: Config schema & public library API (part 1/4) #661

Wave 4 · 20 finding(s) in 20 fix package(s). Fix these whenever a pull request touches this module ("clean the module you touch").

Findings

• CR-A01-071 (code, Low) — _version.py justifies the 1.2.0 api_version bump by calling experimental symbols stable-tier
  severity Low · reachability default · intent contract-violation · confidence likely · independently confirmed · not reproduced (code trace)
  Failure scenario. A maintainer applying the bump rule takes the comment as precedent and treats experimental-tier signature changes as API-versioned.
  Where. forgelm/_version.py:53-66, forgelm/__init__.py:170 (confirmed), forgelm/__init__.py:175 (confirmed), forgelm/_version.py:52
  Recommended fix. Fix the comment, or decide whether these two symbols are stable.

• CR-A01-072 (code, Low) — Secret redaction is done by per-method overrides; repr/str, sub-block model_dump_json and root dumps still expose synthetic.api_key, hf_token, webhook.url and monitoring.endpoint
  severity Low · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A library user logs logger.debug('cfg=%r', cfg) or persists cfg.synthetic.model_dump_json(), and the teacher API key lands in logs or artefacts.
  Where. forgelm/config.py:1452-1458 (SyntheticConfig.model_dump override only), forgelm/config.py:1510-1523 (AuthConfig.hf_token field plus repr and model_dump override; its description claims auto-redaction from log output and serialised mani…, forgelm/config.py:1654-1680 (ForgeConfig.model_dump masks only auth.hf_token and synthetic.api_key), forgelm/config.py:1453-1458
  Recommended fix. Use SecretStr (or a field_serializer plus repr=False) for hf_token, api_key and URL secrets so every serializer path masks them, and accept and route the JSON-only kwargs in model_dump_json.

• CR-A01-073 (code, Low) — Pipeline auto-chain keeps the root model.revision against the previous stage's local output, so stage manifests pair a local dir with the base model's SHA
  severity Low · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An auditor reads stage 2's manifest and sees base model './stage1/final_model' 'pinned' to the commit of the stage-1 Hub base, a claim about a different artefact.
  Where. forgelm/config.py:2223-2231, forgelm/model.py:136-152 (confirmed), forgelm/model.py:566-571, forgelm/config.py:2224-2231 (confirmed)
  Recommended fix. Drop model.revision when auto-chaining or applying input_model_override (or reject a revision on a local path), and record the chain link instead.

• CR-A01-074 (code, Low) — config_template.yaml auth and loss-gate comments describe behaviour that does not exist
  severity Low · reachability default · intent contract-violation · confidence confirmed · independently confirmed · not reproduced (code trace)
  Failure scenario. An operator expects a prompt or a restore and is surprised by a silent unauthenticated run or a deleted model.
  Where. config_template.yaml:173-174, config_template.yaml:179-180, forgelm/utils.py:21-52, forgelm/trainer.py:992-996
  Recommended fix. Reword: 'auth via auth.hf_token, HUGGINGFACE_TOKEN/HF_TOKEN env, or the local HF token file'; 'deletes the final model artefacts'.

• CR-A01-076 (code, Low) — JSON-mode config errors are not logged to stderr before exit
  severity Low · reachability default · intent contract-violation · confidence likely · single reviewer · reproduced end to end
  Failure scenario. A log aggregator that scrapes stderr misses config failures in JSON-mode jobs.
  Where. forgelm/cli/_config_load.py:43-47
  Recommended fix. Call logger.error before printing the envelope in both branches.

• CR-A01-077 (code, Low) — YAML alias expansion through the free-form training.rope_scaling dict gives about 3,400x memory amplification at config hashing
  severity Low · reachability default · intent intended-and-acceptable · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An unreviewed fork-PR config with one more alias level OOM-kills the CI runner or training host before training starts.
  Where. forgelm/config.py:2250-2252 (the safe_load call is on line 2252), forgelm/config.py:572-581, forgelm/config.py:638-711, forgelm/compliance.py:1358-1363
  Recommended fix. Reject unknown keys in rope_scaling (allow-list type/rope_type/factor/short_factor/long_factor/original_max_position_embeddings/...), and/or use a YAML loader that refuses aliases or caps the expanded node count.

• CR-A01-078 (code, Low) — config_template.yaml: the commented synthetic example is rejected when uncommented as written
  severity Low · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A new user uncomments the synthetic section as shown and gets exit 1 on the first run.
  Where. config_template.yaml:103-117, forgelm/config.py:1400-1417
  Recommended fix. Comment out the teacher_revision sample line, or switch the sample to teacher_backend: local.

• CR-A01-080 (code, Low) — Out-of-range dare_seed passes config validation and fails at merge time with exit 2 instead of exit 1
  severity Low · reachability default · intent contract-violation · confidence likely · single reviewer · reproduced end to end
  Failure scenario. dare_seed: 18446744073709551616 -> config valid, then exit 2 at merge time.
  Where. forgelm/config.py:220-223, forgelm/merging.py:354, forgelm/merging.py:439-440, forgelm/cli/_no_train_modes.py:176-177
  Recommended fix. Bound dare_seed in the schema.

• CR-A01-082 (code, Low) — Non-additive change to stable ForgeConfig surface (merge.models dict -> MergeInput, tightened validation) shipped with no api_version classification
  severity Low · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Where. forgelm/config.py:138, forgelm/config.py:198, forgelm/config.py:190, forgelm/config.py:200
  Recommended fix. Make an explicit bump decision for the ForgeConfig change before cutting the release. Either record it as MAJOR (narrowed attribute type, stricter acceptance) or document why it is out of the stable contract. Add the decision to the _version.py bump history and to the CHANGELOG 'Breaking' call-out. Consider keeping a dict-compatible accessor on MergeInput.

• CR-A01-083 (code, Low) — min_valid_fraction docs say 0.0 restores the 'pre-0.11' behaviour; the old behaviour is v0.11.0's, and 0.0 does not fully restore it
  severity Low · reachability default · intent contract-violation · confidence likely · single reviewer · not reproduced (code trace) · already tracked: PARTIALLY-FIXED CORE-04 — docs/roadmap/phase-16-trust-surface-hardening.md:356-380 (S3 checkbox open, delivered by cab2947); CHANGELOG.md:67-74
  Where. forgelm/config.py:1118-1126, docs/reference/configuration.md:232, forgelm/judge.py:610-621
  Recommended fix. Say 'the v0.11.0 behaviour for non-empty eval sets' and note that an empty eval set now always fails.

• CR-A01-084 (code, Low) — The torchrun launch line in config_template.yaml does not work
  severity Low · reachability default · intent contract-violation · confidence likely · independently confirmed · reproduced end to end
  Failure scenario. An operator copies the comment from the canonical template and the multi-GPU launch fails immediately.
  Where. config_template.yaml:164
  Recommended fix. Use torchrun --nproc_per_node=N -m forgelm.cli --config my_config.yaml.

• CR-A01-085 (code, Low) — Six core modules have no module docstring, and several compliance-touching modules do not cite the EU AI Act article
  severity Low · reachability default · intent contract-violation · confidence likely · single reviewer · reproduced end to end
  Failure scenario. help(forgelm.trainer) and the generated API reference show nothing, and reviewers get no statement of ownership or article coverage for the central modules.
  Where. forgelm/config.py:1, forgelm/data.py:1, forgelm/model.py:1, forgelm/trainer.py:1
  Recommended fix. Add one-paragraph module docstrings. Add an AST guard to the gauntlet.

• CR-A01-086 (code, Low) — Upper-case 40-hex revision is accepted as a pin but cannot load from the commit-addressed cache offline
  severity Low · reachability documented-opt-in · intent contract-violation · confidence likely · independently confirmed · reproduced end to end
  Where. forgelm/config.py:47-112, forgelm/config.py:103-112, forgelm/compliance.py:1153-1195, docs/reference/configuration.md:526-530
  Recommended fix. Normalise 40-hex SHAs to lowercase for the load and cache probe, keeping revision_requested verbatim.

• CR-A01-088 (code, Low) — CLI --offline is lost in a pipeline stage that declares its own model block: the stage config records offline=false and an HF login is attempted
  severity Low · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A library or tooling consumer of the materialised stage config sees offline=False and makes Hub calls when it is not running under the CLI's exported env.
  Where. forgelm/config.py:2212-2213 (confirmed), forgelm/cli/_config_load.py:50-57 (confirmed), forgelm/cli/_pipeline.py:1562-1563 (confirmed), forgelm/cli/_pipeline.py:1562
  Recommended fix. In merge_pipeline_stage_config or the orchestrator, OR the root model.offline into every stage's model block.

• CR-A01-089 (code, Low) — webhook.url_env (and monitoring.endpoint_env) are documented to override the inline value, but the code prefers the inline url/endpoint
  severity Low · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An operator rotates a leaked webhook by setting url_env in CI, relying on the documented override, but leaves the old inline url in YAML. Notifications keep going to the old URL.
  Where. forgelm/config.py:1468-1470 (confirmed), forgelm/webhook.py:79-83, tests/test_webhook.py:121-143, forgelm/config.py:1288-1290 (confirmed)
  Recommended fix. Fix the field description to 'used when url is unset' (the code behaviour matches the reference doc) and add a precedence test with both set.

• CR-A01-090 (code, Low) — GaLore layerwise optimiser accepted with default gradient_accumulation_steps=2 and with torchrun/accelerate DDP; fails only inside HF at train time
  severity Low · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Where. forgelm/config.py:1931-1936, forgelm/trainer.py:544-561
  Recommended fix. When a layerwise variant is selected, reject gradient_accumulation_steps != 1 at config time, and word the DDP check as limited to configured strategies (or detect WORLD_SIZE).

• CR-A01-091 (code, Low) — No regression pin for the 'TrainResult importable without torch/transformers' contract
  severity Low · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · not reproduced (code trace)
  Failure scenario. A refactor adds import torch at the top of forgelm/results.py, for example for a tensor-typed metric. All tests still pass, and from forgelm import TrainResult then fails in a torch-less environment, contrary to architecture.md:128.
  Where. tests/test_lazy_imports.py:63-129, forgelm/results.py:1, tests/test_lazy_imports.py:19-29, tests/test_library_api.py:308-326
  Recommended fix. Add _assert_no_eager_heavy_imports('forgelm.results', ('torch','transformers','trl','datasets')).

• CR-A01-092 (code, Low) — setup_authentication token sources are tested, but only with login() mocked, so token persistence, HF_TOKEN/HF_HOME discovery and the false 'No token found' warning are invisible
  severity Low · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced with stubs
  Failure scenario. A CI job or user with HF_TOKEN exported (or HF_HOME relocated), or a gated private repo reachable through the standard HF variables, gets a false 'No Hugging Face token found' warning from ForgeLM, or a login() that persists and activates the token in the user's HF store as a side effect. Both sit behind mocks in every test, so the suite stays green.
  Where. forgelm/utils.py:21-55, tests/test_utils.py:27-117, tests/test_utils.py:79-86, forgelm/cli/_training.py:137
  Recommended fix. Minimal tests that would have caught M-A01-015. (1) Set HF_HOME=tmp, HF_TOKEN='hf_x', clear HUGGINGFACE_TOKEN, and call the real setup_authentication with huggingface_hub's whoami/login stubbed at the HTTP boundary (not forgelm.utils.login). Assert no 'No Hugging Face token' warning, and that tmp/token was not written. (2) Assert that, when a token is supplied through HF_TOKEN or a token file in…

• CR-A01-093 (code, Low) — No test pins secret-free repr/str, sub-block JSON dumps or error text for the five blocks; str(AuthConfig) leaks hf_token; webhook.url and monitoring.endpoint have no redaction test at all
  severity Low · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A future refactor adds logger.debug('config: %s', cfg) or logger.info('%s', cfg.auth), or someone adds cfg.webhook.model_dump_json() to a manifest. The token, webhook url or monitoring key reaches logs or artefacts and no test fails.
  Where. tests/test_config.py:783, tests/test_compliance.py:2285-2325, tests/test_webhook.py:735-750, tests/test_integration.py:123-125
  Recommended fix. Add a parametrized test: for each secret field of ForgeConfig, SyntheticConfig, AuthConfig, WebhookConfig and MonitoringConfig, assert the secret is absent from repr, str, model_dump, model_dump(mode='json'), model_dump_json and str(ValidationError). Add a test that goes through the real ForgeConfig in place of the _FakeConfig classes. Define __str__ alongside __repr__ on AuthConfig, or move…

• CR-A01-094 (code, Low) — Non-UTF-8 or binary config: no test through _load_config_or_exit; the JSON error envelope is only tested for a missing file; the exit-1 outcome is a Python-traceback coincidence
  severity Low · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end · already tracked: KNOWN-OPEN C33-CLI-ERROR-ENVELOPE — docs/roadmap/phase-16-trust-surface-hardening.md:507-508 (S7 open)
  Failure scenario. CI runs forgelm --config cfg.yaml --output-format json on a config saved in CP1252 or UTF-16 (common on Windows editors). It gets an empty stdout, so a JSON parser fails with a different error. The exit code is 1, which the pipeline reads as a config error, but the wrapper has no error field to show.
  Where. forgelm/cli/_config_load.py:26-47, forgelm/config.py (load_config, utf-8 open), tests/test_config_non_finite_matrix.py:106-113, tests/test_cli.py:184-221
  Recommended fix. Catch UnicodeDecodeError (and ValueError from the YAML reader) in _load_config_or_exit and load_config, and add parametrized tests (binary, latin-1, UTF-16, directory, permission denied) that assert SystemExit(1) and, in JSON mode, a parseable {success:false,error} line on stdout. Remove the two dead except branches or cover them.

Done when

• Every finding above is fixed or closed with a reason (tick it here).
• Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}