Repository navigation
Conversation
Closes ghost-features-analysis-20260502 GH-1 (onboarding bloker — 30 doc references across 8 user-manual files no longer point at a non-existent command). The first command an operator should run after installation. Probes: python.version Python is in the supported window torch.cuda torch + CUDA availability gpu.inventory visible GPUs + per-device VRAM extras.<name> every optional extra (qlora, unsloth, deepspeed, evaluation, wandb, mergekit, ingestion, ingestion-pii-ml, ingestion-scale) hf_hub.reachable HF Hub HEAD probe (skipped under --offline) hf_hub.offline_cache local HF cache size + populated state (--offline-only replacement for the Hub probe) disk.workspace free space on the workspace filesystem (<10 GiB fail, <50 GiB warn, otherwise pass) operator.identity FORGELM_OPERATOR audit-identity hint Three statuses: - pass: invariant holds, nothing to do - warn: operator-actionable but doesn't block (missing optional extra, CPU-only torch, FORGELM_OPERATOR unset) - fail: ForgeLM cannot work this way (Python <3.10, no torch, unresolvable operator identity) Exit codes follow the public contract: - 0 (EXIT_SUCCESS): every check passed (warns are OK) - 1 (EXIT_CONFIG_ERROR): at least one fail - 2 (EXIT_TRAINING_ERROR): a probe itself crashed (operator-actionable bug, surfaced as runtime-error class so CI/CD retry logic treats it the same way) Implementation: - forgelm/cli/subcommands/_doctor.py (new, ~430 lines): probes, renderers, exit-code mapper, top-level dispatcher. Heavy deps (torch, huggingface_hub) are imported lazily inside individual check functions so `forgelm doctor` runs on a brand-new machine where torch is not yet installed without crashing. One probe crashing does not abort the rest of the report. - forgelm/cli/_parser.py: _add_doctor_subcommand registrar with --offline + --output-format flags. - forgelm/cli/_dispatch.py: doctor dispatch branch. - forgelm/cli/__init__.py: facade re-exports for tests / monkeypatch. Tests (tests/test_doctor.py — 38 cases, all torch-free): TestPythonVersionCheck (4) Python 3.10 → warn, 3.11 → pass, 3.12 → pass, 3.9 → fail TestTorchCudaCheck (3) no torch → fail; CPU-only torch → warn; torch + CUDA → pass TestGpuInventoryCheck (3) no torch → fail; no CUDA → warn (count=0); 2 GPUs → pass (with VRAM enumeration validated) TestOptionalExtraCheck (2) present module passes; missing module warns with install hint TestHfHubReachableCheck (2) network outage warns (not fails); HTTP 200 passes TestHfCacheOfflineCheck (3) missing cache → warn; populated cache → pass; empty cache → warn TestDiskSpaceCheck (3) free space query smoke; <10 GiB → fail; 30 GiB → warn TestOperatorIdentityCheck (2) FORGELM_OPERATOR set → pass; unset (with getuser fallback) → warn TestExitCodeMapping (4) all-pass → 0; warn-only → 0; any-fail → 1; crashed-probe → 2 TestRenderers (3) text summary line; JSON envelope shape; success=True with no fails TestProbeCrashIsolation (1) one crashing probe does not abort the run; converted to fail with crashed=True marker TestCheckPlan (2) --offline uses cache probe (not Hub probe); all extras in plan TestDispatcher (3) text output prints summary; JSON envelope structure; exit code resolved from result list TestDoctorCLISmoke (2) `forgelm doctor --help` advertises --offline + --output-format; `forgelm doctor --offline --output-format json` end-to-end smoke TestFacadeReExports (1) every test-touched helper reachable via forgelm.cli Documentation: - docs/usermanuals/en/getting-started/first-run.md sample output updated to match the shipped table format. - TR mirror first-run.md updated in the same shape. - CHANGELOG.md [Unreleased] Phase 34 entry. Existing user-manual references (30 across 8 files in installation, first-run, troubleshooting, air-gap, cli reference) were already in place; they now point at a real working command instead of a ghost feature. The closure plan §10 Faz 30 Task A still has the larger doc sweep on its list; this PR ships the minimum doc updates that keep the user-manuals correct in the meantime. Test count: 1013 -> 1051 (+38). Coverage 67.94% -> 68.59%. ruff format/check clean. Closes: closure-plan-202604300906.md §9.5 Phase 34 ghost-features-analysis-20260502.md GH-1 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Sorry @cemililik, you have reached your weekly rate limit of 500000 diff characters.
Please try again later or upgrade to continue using Sourcery
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Review Summary by QodoAdd
WalkthroughsDescription• Implement forgelm doctor subcommand for environment diagnostics - Probes Python version, torch/CUDA, GPU inventory, optional extras - Checks HF Hub reachability and workspace disk space - Validates FORGELM_OPERATOR audit-identity hint • Support --offline mode for air-gapped deployments - Skips network probe, inspects local HF cache instead • Provide text and JSON output formats with structured exit codes - 0 = all pass, 1 = config error, 2 = probe crash • Add 38 comprehensive tests covering all probe scenarios Diagramflowchart LR
A["CLI Parser"] -->|register| B["doctor subcommand"]
B -->|dispatch| C["_run_doctor_cmd"]
C -->|execute| D["Check Plan"]
D -->|run probes| E["Python/Torch/GPU/Extras/Network/Disk/Identity"]
E -->|collect| F["CheckResult List"]
F -->|render| G["Text or JSON Output"]
G -->|resolve| H["Exit Code 0/1/2"]
H -->|exit| I["Process Exit"]
File Changes1. forgelm/cli/subcommands/_doctor.py
|
Code Review by Qodo
1.
|
Not up to standards ⛔🔴 Issues
|
| Category | Results |
|---|---|
| BestPractice | 1 minor |
| ErrorProne | 1 high |
| Security | 1 medium 1 high |
🟢 Metrics 148 complexity · 0 duplication
Metric Results Complexity 148 Duplication 0
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
There was a problem hiding this comment.
Code Review
This pull request introduces the forgelm doctor subcommand, a comprehensive environment diagnostic tool that checks Python versions, PyTorch/CUDA availability, GPU inventory, optional extras, disk space, and HuggingFace Hub connectivity. The changes include CLI integration, documentation updates in multiple languages, and extensive unit testing. Feedback suggests enhancing the HuggingFace probes by respecting the HF_ENDPOINT environment variable and correcting the cache directory resolution logic to properly handle HF_HOME subdirectories and path expansion.
| import urllib.error | ||
| import urllib.request | ||
|
|
||
| request = urllib.request.Request("https://huggingface.co/api/models", method="HEAD") |
There was a problem hiding this comment.
The Hugging Face Hub reachability probe uses a hardcoded URL, which ignores the HF_ENDPOINT environment variable. This can lead to false warnings in enterprise or air-gapped environments where a local mirror is configured via HF_ENDPOINT.
| request = urllib.request.Request("https://huggingface.co/api/models", method="HEAD") | |
| endpoint = os.environ.get("HF_ENDPOINT") or "https://huggingface.co" | |
| url = f"{endpoint.rstrip('/')}/api/models" | |
| request = urllib.request.Request(url, method="HEAD") |
| is a warning — the operator may need to run ``forgelm cache-models`` | ||
| (Phase 35) before training. | ||
| """ | ||
| cache_dir = os.environ.get("HF_HOME") or os.path.expanduser("~/.cache/huggingface/hub") |
There was a problem hiding this comment.
The logic for resolving the Hugging Face cache directory is incorrect when HF_HOME is set. According to the huggingface_hub standard, the hub cache is located in a hub subdirectory of HF_HOME. The current implementation uses HF_HOME directly, which will result in incorrect size calculations (including datasets and other metadata) and may fail to find the actual model cache if it's nested. Additionally, environment variables like HF_HOME should be expanded for user home directory symbols (~) to ensure os.path.isdir works correctly.
cache_dir = os.environ.get("HF_HUB_CACHE")
if not cache_dir:
hf_home = os.environ.get("HF_HOME")
if hf_home:
cache_dir = os.path.join(os.path.expanduser(hf_home), "hub")
else:
cache_dir = os.path.expanduser("~/.cache/huggingface/hub")Closes the two HIGH findings + four bot comments. HIGH fixes: - F-27-01 (HIGH): GH-1 closure was incomplete — only 2 of 10 doc files had been updated to match the shipped output. This commit updates the rest: * docs/usermanuals/en/getting-started/installation.md — sample "what doctor reports" rewritten to enumerate the actual probe set + JSON envelope shape + --offline flag. * docs/usermanuals/en/operations/air-gap.md — sample --offline output replaced with the real table format (was: aspirational text "ForgeLM 0.5.2 (offline mode verified)" + "Llama Guard 3 8B available locally" — none of which the dispatcher actually produces). * docs/usermanuals/tr/* mirrors updated in lock-step. troubleshooting.md was already accurate (only references the command name, no sample output to drift). - F-27-02 (HIGH): HF Hub probe partial migration toward forgelm._http discipline. HF_ENDPOINT env var is now respected via a new _resolve_hf_endpoint() helper (mirrors huggingface_hub's own resolution) so self-hosted mirrors / corp-proxy installs no longer produce false-warn results. Probe carries a dedicated "forgelm-doctor/Phase-34" User-Agent so server logs identify it. Full migration to forgelm._http.safe_get is deferred to XPR-02 cross-PR work (requires adding safe_get to _http.py). Bot fixes (gemini, qodo): - gemini bot: HF cache resolution honours HF_HUB_CACHE > HF_HOME/hub > ~/.cache/huggingface/hub. Was: pointed at HF_HOME directly, which is the parent of multiple cache trees (hub + datasets + spaces) — wrong directory. - gemini bot: HF_ENDPOINT respected (covered above with F-27-02). - qodo bot: _OPTIONAL_EXTRAS list audited against pyproject.toml. Original used aspirational names ("deepspeed", "evaluation", "wandb", "mergekit") that did not match the published extras ("distributed", "eval", "tracking", "merging") — install hints were unactionable. Also added "export" extra to the probe list. - qodo bot: _check_operator_identity now respects FORGELM_ALLOW_ANONYMOUS_OPERATOR=1 the same way AuditLogger does. Without the opt-in: fail (matches AuditLogger's hard-refuse). With the opt-in: warn (matches AuditLogger's "anonymous@host" fallback). Doctor's verdict now matches actual training-time behaviour. MEDIUM fixes: - F-27-03 (MEDIUM perf): _check_hf_cache_offline walk is now depth-capped (4 levels) + file-capped (5 000 files); cap-hit is recorded in extras["walk_truncated"] so downstream tooling can detect partial-scan results. Prevents 30+s scans on NFS-mounted 50+ GiB caches. - F-27-04 (MEDIUM correctness): dropped the dead socket.timeout except branch. socket.timeout is an alias for TimeoutError since Python 3.10 (project floor) and urllib.request.urlopen wraps timeouts as URLError anyway. - F-27-05 (MEDIUM security): added _DOCTOR_SECRET_ENV_NAMES allow- list + _mask_env_value_for_audit helper. FORGELM_OPERATOR is not on the list (operator identity, not secret) but the discipline is pinned for any future probe surfacing HF_TOKEN / FORGELM_AUDIT_SECRET. Tests added (TestHfEndpointResolution, TestOperatorIdentityAnonymousOptIn, TestSecretEnvMasking, plus updated TestHfCacheOfflineCheck for the HF_HUB_CACHE > HF_HOME/hub > default precedence). +7 tests (38 → 45 in test_doctor.py). DEFERRED (out of this PR's scope): - F-27-06 (MEDIUM contract): JSON envelope schema page — docs/reference/cli_json_schemas.md is XPR-03 cross-PR concern. - F-27-07 (MEDIUM): "skip" status for HF Hub probe under --offline — current behaviour swaps the probe rather than emitting a "skip" row; pinning the swap behaviour explicitly is a separate follow-up. - F-27-08..15 LOW/NIT: backlog. Tests: 1051 → 1058 (+7). ruff format/check clean. Coverage 68.59% → 68.74% (+0.15). Closes (review IDs): F-27-01, F-27-02, F-27-03, F-27-04, F-27-05 + gemini-HF_ENDPOINT, gemini-HF_HOME, + qodo-extras, qodo-anonymous-opt-in Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Wave 2a Round-1 multi-agent review — fixes pushedAll CRITICAL + HIGH findings from the multi-agent review (and the relevant inline bot comments) have been addressed in the latest commit on this branch. See the commit message for the per-finding mapping (review IDs F-27-*, gemini bot IDs, qodo bot IDs, sourcery bot IDs). Per-PR test counts:
ruff format/check clean across all five PRs. Cross-PR XPR-01 concern (audit-log parser proliferation) closed in PR #25 via the new Items deferred to follow-up PRs (with rationale) are listed under DEFERRED in each commit message. @coderabbitai review |
|
Tip For best results, initiate chat on the files or code changes.
[review] |
|
|
Superseded by #28 (consolidated Wave 2a integration). All commits from this branch are merged into |
…PR#63-review)
_format_user_assistant_row gated System validation on truthiness, so 0/False/0.0/[]/{} bypassed clean_string's loud-fail contract and were silently dropped. Gate on sys_text != '' so only the genuine empty-string absent-system path skips, and every other value (incl. None) fails loudly. CodeRabbit #27.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>




Summary
Closes the GH-1 onboarding bloker from `ghost-features-analysis-20260502`: 30 doc references across 8 user-manual files (installation, first-run, troubleshooting, air-gap, CLI reference) used to point at a non-existent command. With this PR they now point at a real working subcommand.
`forgelm doctor` is the first command an operator should run after installation. Probes Python, torch, CUDA, GPU inventory, every optional extra advertised in pyproject.toml, HF Hub reachability (or the local cache when `--offline`), workspace disk space, and the FORGELM_OPERATOR audit-identity hint.
```shell
$ forgelm doctor
forgelm doctor — environment check
[✓ pass] python.version Python 3.11.4 (CPython).
[✓ pass] torch.cuda torch 2.4.0 with CUDA 12.4.
[✓ pass] gpu.inventory 1 GPU(s) — GPU0: NVIDIA RTX 4090 (24.0 GiB).
[✓ pass] extras.qlora Installed (module bitsandbytes, purpose: 4-bit / 8-bit QLoRA training).
[! warn] extras.deepspeed Optional extra missing — install with: pip install 'forgelm[deepspeed]' (purpose: DeepSpeed ZeRO + offload distributed training).
[✓ pass] hf_hub.reachable HuggingFace Hub reachable (HTTP 200).
[✓ pass] disk.workspace Workspace /home/me/forgelm — 387.0 GiB free of 500.0 GiB.
[! warn] operator.identity FORGELM_OPERATOR not set; audit events will fall back to 'me@workstation'.
Summary: 7 pass, 2 warn, 0 fail.
```
`--output-format json` returns a structured envelope; `--offline` skips the network probe and inspects the local HF cache instead.
Status semantics
Exit codes follow the public contract: 0 (all pass / warns), 1 (any fail), 2 (a probe itself crashed — runtime-error class).
Files
Test plan
Risk
Low. Strictly additive new subcommand with no shared mutable state; existing subcommands untouched. Heavy deps imported lazily inside check functions so `forgelm doctor` is safe to run on a brand-new machine where torch is not yet installed.
Closes
🤖 Generated with Claude Code