Skip to content

[Wave 4] Low/info — Module sweep: CLI framework: parser, dispatch, exit-code contract, logging (part 3/3) #655

Open

Description

@cemililik

Wave 4 · 9 finding(s) in 9 fix package(s). Fix these whenever a pull request touches this module ("clean the module you touch").

Findings

  • CR-A05-066 (code, Low) — SOP Data Management audit bullet and checklist are stale: quality filter called optional (it is on by default), a 'near-duplicate rate' that is not emitted, and a Croissant card 'alongside' the repor…
    severity Low · reachability default · intent contract-violation · confidence likely · single reviewer · reproduced end to end
    Failure scenario. An operator following the checklist assumes quality_summary is absent unless the flag is passed, or looks for a separate Croissant file and finds none. A reviewer looks for a near-duplicate rate and has to derive one from pair counts, which do not map one-to-one to rows.
    Where. docs/qms/sop_data_management.md:77-78, docs/qms/sop_data_management.md:83-86, docs/qms/sop_data_management.md:103-106, docs/qms/sop_data_management-tr.md:78
    Recommended fix. Say that the quality filter is on by default (opt out with --no-quality-filter), refer to near-duplicate pair counts, and say the Croissant card is embedded under the 'croissant' key. Mirror in TR.

  • CR-A05-067 (code, Low) — --wizard accepts every root mode and pipeline flag, so 'Start training now?' = yes runs a different mode (exit 0) or fails late with exit 1 after the whole interactive session
    severity Low · reachability default · intent intended-but-harmful · confidence likely · independently confirmed · reproduced end to end
    Failure scenario. An operator runs forgelm --wizard --compliance-export ./ev, answers yes to 'Start training now?' and sees 'Running: forgelm --config my.yaml'. No model is trained, empty-metric compliance artefacts are written and the exit code is 0. With --merge or --stage the same session ends in exit 1, which could have been reported before the first prompt.
    Where. forgelm/cli/_wizard.py:93-95, forgelm/cli/_dispatch.py:229-278, forgelm/cli/_no_train_modes.py:264-283, forgelm/cli/_parser.py:1348
    Recommended fix. Before launching the wizard, reject --wizard together with --merge, --generate-data, --stage, --resume-from, --input-model and --force-resume (unless --wizard-start-from is given), for example via the mode group or an explicit check in _maybe_run_wizard. For the flags that stay allowed, pass the active mode to run_wizard_full so the prompt and the 'Running:' echo name the real action and print th…

  • CR-A05-068 (code, Low) — The wizard runs before logging is set up: every forgelm INFO/DEBUG message on wizard paths is dropped, --log-level and --quiet have no effect, and WARNING/ERROR lines reach stderr unformatted through…
    severity Low · reachability default · intent contract-violation · confidence likely · independently confirmed · reproduced end to end
    Failure scenario. An operator runs forgelm --wizard --log-level DEBUG to diagnose a slow inline audit of a 2 GB JSONL. Nothing appears for minutes: progress lines and debug output are dropped. With --quiet, WARNING lines still interleave with the prompts. A CI or log tailer that greps for the '[WARNING] forgelm.wizard' format never sees the validate-on-exit failure.
    Where. forgelm/cli/_dispatch.py:221-225, forgelm/cli/_dispatch.py:240, forgelm/cli/_logging.py:29-33, forgelm/wizard/_orchestrator.py:1185
    Recommended fix. Configure logging before the wizard runs (move the _dispatch.py:238-240 block above line 229, or call _setup_logging inside _maybe_run_wizard). Add a test that drives main(['--wizard','--log-level','DEBUG']) and asserts that an INFO record from the ingest or audit path is emitted in the configured format.

  • CR-A05-069 (code, Info) — main() comment still describes the public exit surface as 0/1/2/3/4
    severity Info · reachability default · intent contract-violation · confidence likely · single reviewer · not reproduced (code trace) · already tracked: KNOWN-OPEN C31-EXIT-CODE-PROSE — docs/roadmap/phase-16-trust-surface-hardening.md:782-785 (S15 open)
    Failure scenario. A maintainer reading main() can believe that 5 and 6 are not public codes.
    Where. forgelm/cli/_dispatch.py:157-164
    Recommended fix. Reference _PUBLIC_EXIT_CODES instead of listing the codes.

  • CR-A05-070 (code, Info) — Purge --check-policy help claims are inaccurate
    severity Info · reachability default · intent contract-violation · confidence likely · independently confirmed · reproduced end to end
    Failure scenario. A pipeline runs purge --check-policy without --config and reads violations:[] as policy-compliant.
    Where. forgelm/cli/_parser.py:1111-1115, forgelm/cli/_parser.py:1144-1150, forgelm/cli/subcommands/_purge.py:1108-1149
    Recommended fix. State the config-error exit in the help; make the empty case explicit (for example checked:false).

  • CR-A05-071 (code, Info) — Enumerated argparse behaviour differences 3.10-3.14 for the constructs _parser.py uses: none change parsed values or exit codes, several change help or error text
    severity Info · reachability default · intent intended-and-acceptable · confidence likely (static only) · independently confirmed · reproduced end to end
    Failure scenario. No contract break found. The only visible effects are on help and error text.
    Where. forgelm/cli/_parser.py:585, forgelm/cli/_parser.py:801, forgelm/cli/_parser.py:888, forgelm/cli/_parser.py:1096
    Recommended fix. No code change needed for these. If 3.14 is added to the matrix, expect only the text differences above.

  • CR-A05-072 (code, Info) — allow_abbrev=True on all 20 parsers lets short unique prefixes enable opt-in / safety-relaxing flags; abbreviation adds no attacker capability but defeats literal-flag policy checks and is unpinned b…
    severity Info · reachability default · intent intended-and-acceptable · confidence likely · single reviewer · reproduced end to end
    Where. forgelm/cli/_parser.py:1276, forgelm/cli/_parser.py:58, forgelm/cli/_parser.py:97, forgelm/cli/_parser.py:131
    Recommended fix. Set allow_abbrev=False on the root parser and on every add_parser call. The CLI is a public CI contract and the docs only ever show long forms, so there is no compatibility cost in-repo. Add one parametrised parser test asserting that a representative prefix of each opt-in flag (--tr, --allow-p, --n, --strip-pattern-n, --r) exits 2. Note for the fix: in argparse the flag is per-parser and is not…

  • CR-A05-073 (code, Info) — forgelm.cli re-exports the verify result types and functions but omits them from all and from any identity pin
    severity Info · reachability undocumented · intent intended-and-acceptable · confidence likely · independently confirmed · reproduced end to end
    Where. forgelm/cli/__init__.py:258-281, forgelm/__init__.py:226-240, forgelm/verify/_gguf.py:32, forgelm/verify/_model_integrity.py:18
    Recommended fix. Decide one way. Either drop the 'public library type/function' comments in cli/init.py, since the same names are public at forgelm.* and the CLI copies exist only as test seams; or add them to all and add a one-line identity test (forgelm.cli.VerifyGgufResult is forgelm.verify.VerifyGgufResult for the six names). The first option is cheaper and consistent with the 'Internal' classificat…

  • CR-A05-074 (code, Info) — forgelm --output-format json chat MODEL is accepted but chat never emits an envelope, not even on failure (rc=2 with empty stdout)
    severity Info · reachability default · intent intended-and-acceptable · confidence likely · independently confirmed · reproduced end to end
    Failure scenario. A CI smoke step runs forgelm --output-format json chat ./model < prompts.txt | jq .success. On a model-load failure stdout is empty and jq fails, and on success stdout is not JSON. Neither case matches the documented envelope contract.
    Where. forgelm/cli/_parser.py:66-67, forgelm/cli/_dispatch.py:114-115, forgelm/cli/subcommands/_chat.py:32-34, docs/usermanuals/en/reference/json-output.md:18
    Recommended fix. Either reject --output-format json for chat at dispatch with exit 1 and a JSON error envelope, or emit {success:false,error} on the chat failure path. Also document chat as an explicit exception in json-output.md (EN+TR).

Done when

  • Every finding above is fixed or closed with a reason (tick it here).
  • Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.

Activity

  1. added
    bugSomething isn't working
    backlog: knownAlready tracked in docs/roadmap
    securitySecurity-relevant
    severity: lowMinor defect, inaccuracy or rough edge
    source: review-2026-10Found by the October 2026 code and documentation review
    wave: 4Roadmap wave 4: low / info / watch, when touching the module
    on Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: cliCode area: clibacklog: knownAlready tracked in docs/roadmapbugSomething isn't workingsecuritySecurity-relevantseverity: lowMinor defect, inaccuracy or rough edgesource: review-2026-10Found by the October 2026 code and documentation reviewwave: 4Roadmap wave 4: low / info / watch, when touching the module

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions