Skip to content

[Wave 3] Medium — Module sweep: Dev tooling: documentation, site & CLI-help CI guards #550

Open

Description

@cemililik

Wave 3 · 3 finding(s) in 3 fix package(s). Fix these whenever a pull request touches this module ("clean the module you touch").

Findings

  • CR-A17a-004 (code, Medium) — troubleshooting.md (EN+TR) advises non-existent training.gradient_checkpointing; YAML/code-block guards see only column-0 ```yaml fences and do not schema-check fragments outside docs/usermanuals
    severity Medium · reachability default · intent `contract-violation` · confidence confirmed · independently confirmed · reproduced end to end · already tracked: KNOWN-OPEN C31-DOC-SCHEMA-PROSE — docs/roadmap/phase-16-trust-surface-hardening.md:785-786; docs/roadmap/risks-and-decisions.md:184 (C-9)
    Failure scenario. An operator adds 'gradient_checkpointing: true' under training:. TrainingConfig(**{'gradient_checkpointing': True}) raises extra_forbidden on ('gradient_checkpointing',).
    Where. `docs/guides/troubleshooting.md:163`, `docs/guides/troubleshooting-tr.md:170`, `tools/check_bilingual_code_blocks.py:71-72` (confirmed), `tools/check_yaml_snippets.py:78-85`
    Recommended fix. Remove or replace the snippet in both mirrors and the fit_check recommendation (or add a real field). Extend fragment key-path validation (as in check_usermanual_schema_drift) to docs/guides and docs/reference. Make the code-block extractor indentation- and tilde-aware.

  • CR-A17a-005 (code, Medium) — check_cli_help_consistency skips every top-level 'forgelm --flag' invocation; top_level_flags is dead data and a shipped --output-dir doc error passes
    severity Medium · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
    Failure scenario. An operator follows the GPU-cost calibration recipe and the first command exits with an argparse error. Any future renamed or removed main-CLI flag (--fit-check, --merge, --benchmark-only, ...) cited in docs also passes CI unnoticed.
    Where. tools/check_cli_help_consistency.py:475-479, tools/check_cli_help_consistency.py:210 is a local variable feeding :229, not a separate reader, tools/check_cli_help_consistency.py:229 (confirmed), tools/check_cli_help_consistency.py:694
    Recommended fix. Validate top-level invocations against the top-level usage block, not the whole help text with its epilog. Treat a leading '--' as a top-level form, and fix gpu-cost.md:84 (EN and TR) to use output settings that exist.

  • CR-A17a-006 (code, Medium) — check_cli_help_consistency builds its parser surface lexically from help text: 16 prose words become subcommands and prose-mentioned flags become valid
    severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
    Failure scenario. A doc example forgelm audit data.jsonl --pii-mask (a flag that exists only on ingest) passes CI because audit's help prose mentions --pii-mask, and the user gets an argparse error.
    Where. tools/check_cli_help_consistency.py:164-199 (confirmed), tools/check_cli_help_consistency.py:153, tools/check_cli_help_consistency.py:123-141 (confirmed), tools/check_cli_help_consistency.py:202-231 (confirmed)
    Recommended fix. Take subcommand names from the usage line's {a,b,...} choice set or from argparse introspection, and harvest flags only from the options section's leading option strings.

Done when

  • Every finding above is fixed or closed with a reason (tick it here).
  • Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.

Activity

  1. added
    bugSomething isn't working
    backlog: knownAlready tracked in docs/roadmap
    severity: mediumReal defect with limited reach or a workaround
    source: review-2026-10Found by the October 2026 code and documentation review
    wave: 3Roadmap wave 3: medium severity, module sweeps
    on Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: dev-toolingCode area: dev-toolingbacklog: knownAlready tracked in docs/roadmapbugSomething isn't workingseverity: mediumReal defect with limited reach or a workaroundsource: review-2026-10Found by the October 2026 code and documentation reviewwave: 3Roadmap wave 3: medium severity, module sweeps

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions