Skip to content

Feature/integrated smoke test - #300

Open
LazyKaddu wants to merge 4 commits into
StellarResearch:mainfrom
LazyKaddu:feature/integrated-smoke-test
Open

LazyKaddu wants to merge 4 commits into
StellarResearch:mainfrom
LazyKaddu:feature/integrated-smoke-test

Conversation

@LazyKaddu

@LazyKaddu LazyKaddu commented Oct 2, 2026 •

Copy link
Copy Markdown

Summary of Changes

  • Created the Core Testing Pipeline
    shift_runner.py: Added the AdaptiveShiftRunner which encapsulates the complete RL research workflow (Training, Freezing, Evaluation, Shock Window, Forking into Fixed vs. Adaptive Arms, Online Adaptation Blocks, and Recovery Analysis).
    test_research_protocol_smoke.py: Created a fast, deterministic pytest that executes this runner with miniature configuration limits (e.g., training_steps=128, bounded evaluation steps) to ensure it can run rapidly in CI pipelines.
  • Squashed Environment & Protocol Bugs
    Missing Environment Fix: Changed the target environment from gridworld (which wasn't registered in the repository or Gymnasium) to the supported drone environment to fix the RegistryError.
    Policy Attribute Fix: Resolved an AttributeError in shift_runner.py where the code was incorrectly trying to access algo.policy.state_dict(). We corrected it to access the underlying Stable-Baselines3 model properly (algo.model.policy.state_dict()).
  • Debugged Windows-Specific CI Test Failures
    Discovered and fixed a tricky Windows-specific bug in tests/test_cli.py. The tmp_path paths were generating Windows backslashes (C:\Users...), causing the YAML parser to crash by mistaking \U for a malformed Unicode escape sequence.
    We fixed this by appending .as_posix() to the paths, ensuring they always use forward slashes safely across all operating systems.

Verification

  • pytest tests/ passes
  • ruff check src/ tests/ passes
  • mypy src/ passes

@LazyKaddu

Copy link
Copy Markdown
Author

Hi @ashishsinghbora waiting for the review for this pr

@ashishsinghbora ashishsinghbora left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@LazyKaddu I went through this PR in detail. The integrated smoke test is a good addition, but I don't think this is merge-ready yet. The main concern is that several of the claimed “protocol invariants” are testing implementation behavior rather than proving that the research protocol is actually being executed correctly.

The biggest issue is the shock-window implementation. "shock_returns" are generated using the same environment and same policy as the pre-shift evaluation, but there is no actual shift/disturbance introduced between the baseline and shock phases. That means the test is calling this a “Shock Window” without actually testing a distribution/environment shift. For an Adaptive RL protocol, this is a critical semantic gap.

The Fixed vs Adaptive comparison is also weaker than it appears. Both arms are initialized from the same frozen weights, which is correct, but the Fixed arm is never actually evaluated as an independent experimental trajectory during the shock window. Instead, both "fixed_post_returns" and "adaptive_post_returns" are initialized with "list(shock_returns)". This makes the assertion that both arms have identical initial shock returns effectively tautological rather than an independently verified invariant.

The checksum validation is also not validating an artifact checksum. The manifest computes a SHA-256 hash over the manifest contents and then stores that hash inside the same manifest. The test only checks that the resulting value has 64 characters. It never recomputes the checksum from persisted output or verifies an actual artifact. So the test currently proves “a SHA-256-looking string was generated,” not artifact integrity.

Another concern is the policy mutation assertion:

"adaptive_l2_delta > 0.0"

This proves that some weights changed, but it does not prove that the online adaptation behaved correctly. Conversely, "fixed_l2_delta == 0.0" is a useful invariant, but it should ideally be enforced against the actual frozen policy used for evaluation, not only against the locally copied SB3 model state.

The runner also creates multiple PPO instances sharing the same "env" object. That makes the experimental lifecycle harder to reason about and can introduce state contamination between training and evaluation. For a research protocol, the environment lifecycle should be explicit: separate training/evaluation environments or controlled resets with clearly defined shift state.

There is also a significant determinism problem. The PR describes the test as deterministic, but the runner performs PPO training without explicitly controlling seeds for the environment, model, NumPy/PyTorch, and action/evaluation state. A short training run can pass today and fail later due to stochastic initialization or optimization behavior. “Fast” and “deterministic” are different properties; this test needs to guarantee the latter.

The test itself also checks several things that are mostly structural rather than semantic. For example, checking "len(fingerprint) == 64" does not establish that the fingerprint corresponds to the expected policy, and checking that recovery values are "int" does not establish that "T_H" was computed correctly.

Finally, the runner is doing a lot of protocol orchestration directly inside a new experimental module. Before merging this, I'd want the protocol phases and their invariants to be clearly separated so that the smoke test exercises the same production protocol implementation rather than becoming a second implementation of the protocol.

My recommendation is Request changes.

The highest-priority fixes are:

  1. Introduce an actual controlled environment shift/disturbance during the shock window.
  2. Execute Fixed and Adaptive arms through independently controlled evaluation trajectories.
  3. Make the test genuinely deterministic with explicit seeds.
  4. Validate persisted artifact/manifest checksums by recomputation.
  5. Strengthen the assertions around recovery correctness instead of only checking types/presence.
  6. Avoid shared environment state between training and independent evaluation phases.

The smoke test is a good direction, but right now it can pass while important parts of the actual Adaptive RL research protocol are not being tested.

@LazyKaddu

Copy link
Copy Markdown
Author

@ashishsinghbora can you now check the revised version of the smoke test that i have send

@AryanXCode646

Copy link
Copy Markdown
Contributor

Hi @LazyKaddu

Thanks for the follow-up revisions. The separate fixed/adaptive environments and explicit shift configuration are steps in the right direction. However, after reviewing the latest changes, I found several outstanding correctness issues that need to be resolved before this PR is ready to merge.

Remaining issues

1. P1 — Duplicate AdaptiveShiftRunner definition

src/adaptive_rl/experiments/shift_runner.py now defines AdaptiveShiftRunner twice. The later definition shadows the newly introduced smoke-test runner, so the class imported by tests/test_research_protocol_smoke.py does not expose the constructor arguments used by the test (env_name, training_steps, k_pre, w, h, block_steps, and seed). This should cause a TypeError before the intended assertions execute.

Please remove the naming collision by either giving the smoke-test helper a distinct name or integrating the workflow into the existing production runner. Avoid maintaining two competing implementations of the same protocol.

2. P1 — Manifest checksum generation and verification are inconsistent

The runner calculates the checksum from a serialized manifest containing "checksum": "", while the test removes the checksum field entirely before recomputing the digest. These are different payloads, so the expected checksum will not match.

Please establish one canonical serialization procedure and use it for both checksum generation and verification. Add a regression test that passes against the persisted manifest, including after serialization and deserialization.

3. P1 — Artifact integrity is not fully verified

The current checksum covers manifest metadata, not the actual bytes of the persisted frozen_policy.zip checkpoint. The policy fingerprint comparison is useful, but it does not replace a file-level integrity check.

Please record the SHA-256 digest of the persisted checkpoint and independently recompute it during verification. Keep the manifest checksum and artifact checksum as clearly defined, separate checks.

4. P1 — Recovery assertions are self-referential

The test invokes compute_recovery() in the runner and invokes the same function again to verify the resulting recovery times. This confirms consistency with the same implementation, but it cannot detect an error in the recovery algorithm itself.

Please add deterministic fixtures with independently calculated expected recovery times. Cover immediate recovery, delayed recovery, failure to recover within the horizon, and persistence/window boundary cases.

5. P2 — Smoke-test return values and seed handling need clearer semantics

_eval_algo() stops after five environment steps and resets every episode with the same seed. The resulting reward sums are truncated evaluation proxies, not necessarily full-episode returns. Using them for recovery calculations without clearly limiting the interpretation risks validating a different metric from the production protocol.

Please either use the production evaluation routine or explicitly document and test this limited smoke-test behavior. Use a reproducible seed schedule appropriate to the intended evaluation coverage, and assert that the nominal and shifted environments differ in the intended, effective dynamics.

6. P2 — Documentation and validation coverage do not match

docs/TESTING.md claims that claim validation passes on generated output, but the new smoke test does not visibly invoke the corresponding claim validator. Please execute the actual validator and assert its outcome, or correct the documentation to reflect what the test really verifies.

Required verification before requesting another review

Please address the constructor/name collision first, then fix the checksum contract and strengthen the remaining assertions. Run at least:

  • pytest -v tests/test_research_protocol_smoke.py
  • The relevant existing regression and CLI tests.
  • ruff check src/ tests/
  • mypy src/

The current Checks page reports a workflow run completed with no jobs, so it does not establish that the new smoke test has actually passed. Please ensure the applicable CI jobs execute and provide their results.

The feature is a useful addition, but a smoke test must verify the intended workflow rather than merely contain assertions for it. I recommend keeping this PR in Changes Requested until the blocking issues are fixed, the regression tests pass, and the CI results are verifiable.

Thanks for addressing these points.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants