Skip to content

feat(env): publish task-registered custom step-output carriers through the persistent-step-buffer ring - #2631

Merged
TATP-233 merged 3 commits into
mainfrom
feat/issue-2628-persistent-step-carriers
Nov 19, 2026
Merged

TATP-233 merged 3 commits into
mainfrom
feat/issue-2628-persistent-step-carriers

Conversation

@TATP-233

Copy link
Copy Markdown
Collaborator

Summary

  • Adds TorchEnv.register_step_carrier(name, carrier): tasks register a device tensor they update in place during update_state (for example a transition snapshot). With persistent_step_buffers (implied by autoreset_cuda_graph) the env copies it into the current publication-ring slot at every step boundary with pre-allocated, empty_like slot buffers — so consumers hold the published zero-copy view in TorchEnvState.carriers for published_buffer_depth further steps instead of defending with per-step detach().clone().
  • TorchEnvState gains a carriers field; ring depth 0 publishes the working carriers themselves (zero-copy).
  • Registration is cold-path and fails closed: after the first reset (ring allocated), without the fixed-shape chain, on duplicate/empty names, and on non-tensor or wrong-device carriers.
  • Why: issue step 输出载体持久化:persistent_step_buffers 应扩展到 reward/termination/transition 载体,消除 per-step clone 风暴 #2628. reward/terminated/truncated already publish through the ring, but task transition snapshots had no first-class carrier — downstream tasks paid per-step clone kernels plus allocator churn, and per-step-changing addresses block the compute-segment CUDA graph boundary requested in Manager compute 段(step 内 obs/reward/termination/command 编排)在 launch-bound 场景下成为瓶颈:请求融合或整段 CUDA graph 化 #2627. Every address involved is now fixed after the first publish, which is the address-stability precondition for that capture.
  • User-facing behavior: unchanged unless a task opts in by calling register_step_carrier; the step-output lifetime contract (what is ring-persistent vs per-step fresh) is now documented.

Linked Work

Related ADR context: no new structural decision — this completes the carrier contract #2605's ring started. ADR-0238's rejected state-update graph pre-registration already records why fixed carrier addresses are the capture precondition; this PR is the missing step-output side of that contract, not a revival of the rejected boundary.

Validation

  • make test-all passed on the final local head before this PR was created or updated
  • Additional task-specific validation listed below

Commands actually run (final head 810692b2, Linux, Python 3.13, CUDA available):

uv sync --frozen --all-extras
make check
uv run pytest tests/envs -q                                  # 442 passed, 14 skipped
uv run pytest tests/envs/test_step_output_carriers.py tests/envs/test_fixed_shape_autoreset.py -q  # 35 passed (incl. CUDA graph cases)
uv run pytest tests/scripts/test_check_docs.py -q            # 20 passed
make test-all                                                # check + test-cov + benchmark smoke, all green

Note: tests/envs/test_multi_entity_consumer.py initially failed in the fresh worktree venv with ModuleNotFoundError: mjbatch; that was a missing mujoco extra in the new venv, not a code issue — after uv sync --all-extras all four pass.

Remote CI route:

  • Base main: not scheduled yet at PR creation; recording the current-head CI link and status here once the run exists.

Impact

Artifacts

  • W&B: none
  • benchmark result: none (no hot-path change for tasks that do not register carriers; the publish loop only iterates the registered-carrier dict)
  • video / screenshot: none
  • ONNX / checkpoint: none

Checklist

  • Added or updated tests where needed
  • Updated docs if behavior or workflow changed
  • Linked the driving issue
  • Noted any follow-up work explicitly

Follow-up (out of scope here): #2627 compute-segment CUDA graph capture; per-slot final_observation persistence (currently documented as aliasing the working scratch — persisting it per ring slot would add per-step obs-width copies on the fixed-shape path, so it was left as a documented short-lived output).

TATP-233 and others added 3 commits November 19, 2026 23:33
…h the persistent-step-buffer ring

Issue #2628: reward/terminated/truncated already publish through the
persistent-step-buffer ring, but task transition snapshots had no
first-class carrier, so downstream tasks defended against overwrites
with per-step detach().clone() and could not rely on fixed addresses.

TorchEnv.register_step_carrier registers a task-owned device tensor as
a step-output carrier. The ring allocates per-slot buffers with
empty_like at the first publish and copies the working carrier into
the current slot at every step boundary, so consumers hold the
published zero-copy view for published_buffer_depth further steps.
Registration is cold-path and fails closed (after the first publish,
without the fixed-shape chain, duplicate/empty names, non-tensor or
wrong-device carriers). TorchEnvState gains a carriers field; depth 0
publishes the working carriers themselves.

The sync-free autoreset user guide documents the step-output lifetime
contract, including which outputs remain per-step fresh (info,
final_observation).
@TATP-233
TATP-233 merged commit 2227c91 into main Nov 19, 2026
4 of 10 checks passed
@TATP-233
TATP-233 deleted the feat/issue-2628-persistent-step-carriers branch November 19, 2026 15:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant