Skip to content

[Wave 3] Medium — Module sweep: Operator subcommands: doctor, cache-models, cache-tasks #564

Open

Description

@cemililik

Wave 3 · 9 finding(s) in 8 fix package(s). Fix these whenever a pull request touches this module ("clean the module you touch").

Findings

  • CR-A15-004 (code, Medium) — Doctor --offline cache probe passes on any regular file (audit files, locks, .incomplete blobs) and warns with exit 0 on a missing or empty cache, so the documented air-gap gate gives false assurance
    severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
    Failure scenario. Staging host: cache-models fails (or the bundle is truncated in transfer). Target host: CI step 'forgelm doctor --offline --output-format json -q | jq -e .success' is green, training then fails 90 minutes in on a missing snapshot - the exact outcome the guide says doctor prevents.
    Where. forgelm/cli/subcommands/_doctor.py:699-708, forgelm/cli/subcommands/_doctor.py:708, forgelm/cli/subcommands/_doctor.py:741-748 (confirmed), forgelm/cli/subcommands/_cache.py:286 (confirmed)
    Recommended fix. Count only model snapshot content (models--/snapshots/), exclude audit_log.jsonl*, and make 'no snapshot found' and 'cache dir missing' a fail in --offline mode (or accept --require-model IDs). Keep the audit log out of the Hub cache dir by default. Align the guide wording.

  • ↳ same root cause: DOC-D2B05-011 (docs, Medium) — docs/guides/air_gap_deployment.md: misleading — doctor --offline verifies the bundle survived transfer / validates presence of the snapshot; a detached mount reports fail
    severity Medium
    Where. docs/guides/air_gap_deployment.md:13,129,209 · type: misleading
    The document says. doctor --offline verifies the bundle survived transfer / validates presence of the snapshot; a detached mount reports fail
    What the code does. Only a file count under the resolved cache; missing dir and empty dir are warn (exit 0); a cache with only audit_log.jsonl passes.
    Code references. forgelm/cli/subcommands/_doctor.py:689-763
    Proposed correction. Fix the probe (F1) or state that it does not verify content or completeness.
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • CR-A15-006 (code, Medium) — cache-models/cache-tasks _maybe_audit_logger swallows every AuditLogger init failure at DEBUG and the run exits 0 with no audit chain
    severity Medium · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced with stubs
    Failure scenario. A compliance-driven operator passes --audit-dir /mnt/audit (read-only or a typo). The bundle is staged and exit 0 is reported, but no Art. 12 record of who staged it exists.
    Where. forgelm/cli/subcommands/_cache.py:605-625 (debug-only swallow at :620-625; docstring at :606 says 'warn'), forgelm/cli/subcommands/_cache.py:286 and :292-293 (models REQUESTED emission, unguarded), forgelm/cli/subcommands/_cache.py:446-449 (tasks), forgelm/cli/subcommands/_cache.py:286-293
    Recommended fix. Match reverse-pii: an explicit --audit-dir failure is fatal (exit 2). The implicit default may stay best-effort but should warn at WARNING level.

  • CR-A15-008 (code, Medium) — cache-models cannot stage a pinned revision: snapshot_download is called with no revision and no allow_patterns
    severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · not reproduced (code trace)
    Failure scenario. A regulated run pins model.revision to a SHA from three months ago. The operator stages with cache-models (which fetches today's HEAD) and transfers the bundle, and the offline load of the pinned SHA fails on the target host.
    Where. forgelm/cli/subcommands/_cache.py:338 (snapshot_download_callable(repo_id=name, cache_dir=cache_dir), no revision / allow_patterns), forgelm/cli/_parser.py:972-1020 (cache-models parser: --model, --safety, --output, --audit-dir only), forgelm/config.py:272-279 (model.revision, 40-hex recommended), forgelm/model.py:96-160 (prepare_revision_pin passes requested literal as revision= when no SHA can be confirmed offline)
    Recommended fix. Support revision pins in cache-models and record the resolved commit in the envelope and audit event.

  • CR-A15-010 (code, Medium) — Doctor hf_hub.reachable maps every HttpSafetyError to fail (exit 1), including DNS-resolution failures and private-address corporate mirrors
    severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
    Failure scenario. An operator runs forgelm doctor on a laptop with no network, or in a CI job whose runner has no DNS. The doctor exits 1 with success:false and tells them to change HF_ENDPOINT, although the endpoint is already the public default. The documented contract says this case is a warn with exit 0.
    Where. forgelm/cli/subcommands/_doctor.py:499-520 (HttpSafetyError branch is at 507-520), forgelm/_http.py:285-288 (gaierror to block_reason), forgelm/_http.py:707-711 (raise HttpSafetyError), tests/test_doctor.py:540-552
    Recommended fix. Treat the HttpSafetyError whose reason starts with "DNS resolution failed" or "no public IP resolved" as a transport warn. Alternatively, have _resolve_safe_destination signal DNS failure through a distinct exception subclass. Change the unit test to drive the real safe_get with a stubbed socket.getaddrinfo.

  • CR-A15-013 (code, Medium) — Staging the gated Llama Guard repo needs HF_TOKEN at build time; the recipe gives no secret-mount guidance and the docs' only token advice applies to runtime
    severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · not reproduced (code trace)
    Failure scenario. Recipe as written: the build fails at step 3 with exit 2 (gated repo). The operator then 'fixes' it with ARG HF_TOKEN + ENV HF_TOKEN=$HF_TOKEN and ships forgelm-airgap.tar.gz, with the token readable through docker inspect / docker history on the air-gapped host.
    Where. docs/usermanuals/en/operations/docker.md:137-139, docs/usermanuals/en/operations/docker.md:118-125, docs/usermanuals/tr/operations/docker.md:137-139, forgelm/cli/subcommands/_cache.py:297-315
    Recommended fix. Document # syntax=docker/dockerfile:1 + RUN --mount=type=secret,id=hf_token HF_TOKEN=$(cat /run/secrets/hf_token) forgelm cache-models --safety ... with docker build --secret id=hf_token,env=HF_TOKEN, and add a pitfall that names HF_HOME/token as a leak path.

  • CR-A15-014 (code, Medium) — Compliance mappings cite a 'forgelm doctor resource report' for capacity planning; doctor reports only total GPU VRAM and free disk in the cwd
    severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
    Failure scenario. An auditor checking A1.1 or A.8.6 evidence asks for the 'doctor resource report' and gets a pass/warn/fail triage with no memory, CPU or utilisation data. The capacity evidence is not there.
    Where. docs/reference/soc2_trust_criteria_mapping.md:69, docs/reference/soc2_trust_criteria_mapping-tr.md:68, docs/reference/iso27001_control_mapping.md:107, docs/reference/iso27001_control_mapping-tr.md:106
    Recommended fix. Reword to 'forgelm doctor GPU VRAM inventory and workspace free-disk check' or similar, or add RAM/CPU rows to doctor.

  • CR-A15-015 (code, Medium) — Doctor credited as a package inventory (A.8.19) and as a supply-chain treatment (RTP R-02); it inventories no packages and checks no integrity
    severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
    Failure scenario. An ISO auditor who asks for the A.8.19 software-installation record gets doctor output with no installed-version list for the training stack. The R-02 residual-risk rating depends partly on a control that does nothing against a compromised PyPI release.
    Where. docs/reference/iso27001_control_mapping.md:120, docs/reference/iso27001_control_mapping-tr.md:119, docs/qms/statement_of_applicability.md:127, docs/qms/statement_of_applicability-tr.md:128
    Recommended fix. Reword A.8.19 to cite the SBOM and 'pip freeze'/'pip list' (or add a package-version table to doctor), change 'pins' to 'bounded version ranges', and remove doctor from the R-02 treatments, or state that it adds nothing against compromise.

  • CR-A15-016 (code, Medium) — Doctor misreports a broken torch import: a missing CUDA library crashes three probes with exit 2 ('doctor bug'), and an undefined-symbol ImportError is called 'torch is not installed'
    severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced with stubs
    Failure scenario. A host with a CUDA wheel and a missing cuDNN runs doctor as the docs advise. CI sees exit 2, classed as a doctor runtime bug, and its retry branch fires instead of the misconfiguration branch. A host with a torch/CUDA symbol mismatch is told to 'pip install forgelm', which re-pulls the same broken torch.
    Where. forgelm/cli/subcommands/_doctor.py:256-258, forgelm/cli/subcommands/_doctor.py:366-368, forgelm/cli/_abi_check.py:78-81, forgelm/cli/subcommands/_doctor.py:1036-1058
    Recommended fix. Catch OSError as well, and keep the exception text. When the module spec exists (importlib.util.find_spec('torch')) but the import fails, report 'torch is installed but failed to import: ' with the reinstall-matching-CUDA hint. Make numpy.torch_abi skip with reason 'torch_import_failed' rather than 'torch_missing'.

Done when

  • Every finding above is fixed or closed with a reason (tick it here).
  • Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.

Activity

  1. added
    bugSomething isn't working
    documentationImprovements or additions to documentation
    severity: mediumReal defect with limited reach or a workaround
    source: review-2026-10Found by the October 2026 code and documentation review
    wave: 3Roadmap wave 3: medium severity, module sweeps
    on Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: operator-cmdsCode area: operator-cmdsbugSomething isn't workingdocumentationImprovements or additions to documentationseverity: mediumReal defect with limited reach or a workaroundsource: review-2026-10Found by the October 2026 code and documentation reviewwave: 3Roadmap wave 3: medium severity, module sweeps

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions