Repository navigation
[Feature/adaptive rl doc] created the cli for the doctor which checks the dependencies and hardware requirements for the ARL to work on the system - #302
Conversation
|
Hi @ashishsinghbora waiting for the review this pr |
ashishsinghbora
left a comment
There was a problem hiding this comment.
@LazyKaddu I went through this PR in detail. The overall idea of a "system doctor" is useful, but I don't think this is merge-ready yet. The main issue is that several diagnostics look stronger than what they actually validate.
The biggest concern is project-context detection. "check_git_health()", "check_io_permissions()", and "check_yaml_configs()" all operate relative to the current working directory. That means "adaptive-rl doctor" can produce completely different results depending on where the user invokes it from. Running the command outside the repository can report missing "configs/", create "artifacts/" and "logs/" in the user's current directory, and report the wrong Git state. A system diagnostic command should resolve the actual ARL project/package root rather than assuming "cwd == repository root".
The classical planner check is also misleading. "check_classical_planner()" implements a tiny local A* function inside the diagnostic itself. It never imports or executes ARL's actual planner implementation. Therefore, this check can pass even if the real planner in "adaptive_rl/algorithms" is completely broken. This should test the project's real planner with a minimal deterministic fixture.
There is a similar issue with Gymnasium registration. The code attempts to import "adaptive_rl.environments", but deliberately ignores an "ImportError" and continues checking the registry. That can hide the exact failure that prevented environment registration. A failed environment import should be surfaced as a diagnostic failure with the underlying exception, not silently swallowed.
The hardware check also needs clearer semantics. Treating “no CUDA/MPS” as a warning is reasonable if CPU execution is officially supported, but the doctor should determine whether the selected ARL configuration actually requires GPU acceleration. Otherwise a perfectly valid CPU-only deployment can be presented as unhealthy.
The remediation commands are another concern. Hard-coded commands such as "pip install torch" or a specific CUDA wheel can easily produce an environment that conflicts with the project's pinned dependencies. The doctor should preferably reference the project's supported installation mechanism/version constraints instead of inventing a generic pip command.
Finally, the tests are heavily mocked and mostly verify that mocked dependencies return the expected status. They don't give much confidence that the doctor works against a real ARL installation. I'd strongly recommend integration tests covering at least: running from the repository root, running from another directory, missing/broken environment registration, real planner execution, malformed YAML, CPU-only machines, and missing optional dependencies.
So my recommendation is Request changes. The architecture is a good start, but before merging, the doctor needs to prove that it is diagnosing the actual ARL system, not just successfully executing a collection of isolated sanity checks.
|
Hey @ashishsinghbora you can check the changes i have created according to the suggestions you have given |
|
Hi @LazyKaddu Remaining issues1. P1 — CI lint and formatting checks are still failing The latest visible CI run reports that “Check code formatting with ruff” failed. A passing Python 3.10 test job does not compensate for a failing required quality check. Please format the changed files, resolve any remaining lint/type-check issues, and rerun the complete relevant quality checks. Please share the results from the updated commit rather than relying on the verification claims in the PR description. 2. P1 — The classical planner check still needs real integration coverage
Please verify the import path against the real repository implementation and execute the planner using a deterministic fixture without replacing the planner module with a mock. The check must fail meaningfully if the actual planner is missing or broken. 3. P1 — Dependency checks do not align with the project's declared dependencies
Please align the diagnostic checks with 4. P1 — Project-root resolution must support the installation modes the CLI claims to support
In that scenario, Git/configuration checks can inspect the wrong location, and the I/O diagnostic may attempt to create Please distinguish a real checkout from an installed package, avoid assuming that package installation implies the repository is present, and handle unavailable project-level checks explicitly. Add coverage for invocation from another working directory and for the supported installation layouts. 5. P1 — The I/O permission probe can delete an existing file
Please use a uniquely created temporary file with guaranteed cleanup and ensure the probe never removes a pre-existing file. Also verify that temporary filesystem changes are handled safely when a probe fails partway through execution. 6. P2 — YAML validation and automated tests need stronger coverage
Please make the diagnostic's scope explicit and add focused regression or integration tests for malformed YAML, valid YAML with invalid configuration structure, optional dependency combinations, actual planner execution, and execution outside the repository root. Where full schema validation already exists in ARL, reuse it rather than introducing a second implementation. Required verificationBefore requesting another review, please:
The objective is to ensure that the doctor diagnoses the actual installed ARL environment, gives accurate remediation advice, and remains safe to run on a user's machine. Please address the blockers, add regression coverage for the identified cases, and report any intentionally unsupported scenarios explicitly. I recommend keeping this PR in Changes Requested until the outstanding correctness and validation issues are resolved and the relevant CI checks are green. Thanks for continuing to work through the feedback. |
Resolves #290
Summary of Changes
SystemDoctorinsrc/adaptive_rl/diagnostics/doctor.pyto evaluate 10 distinct categories of local environment health.adaptive-rl doctorcommand insrc/adaptive_rl/cli.pyusing Typer.--jsonflag to export a machine-readable structural diagnostic report, and wired the command to return a non-zero exit code (sys.exit(1)) on critical failures.tests/test_doctor.pyproviding isolated unit tests usingunittest.mockto verify the diagnostic logic, JSON schema, and Typer CLI exit codes.README.mdand introduceddocs/GETTING_STARTED.mdto prominently document the system doctor as a first-step troubleshooting tool.Verification
pytest tests/passesruff check src/ tests/passesmypy src/passes