Repository navigation
feat(env): publish task-registered custom step-output carriers through the persistent-step-buffer ring - #2631
Merged
Conversation
…h the persistent-step-buffer ring Issue #2628: reward/terminated/truncated already publish through the persistent-step-buffer ring, but task transition snapshots had no first-class carrier, so downstream tasks defended against overwrites with per-step detach().clone() and could not rely on fixed addresses. TorchEnv.register_step_carrier registers a task-owned device tensor as a step-output carrier. The ring allocates per-slot buffers with empty_like at the first publish and copies the working carrier into the current slot at every step boundary, so consumers hold the published zero-copy view for published_buffer_depth further steps. Registration is cold-path and fails closed (after the first publish, without the fixed-shape chain, duplicate/empty names, non-tensor or wrong-device carriers). TorchEnvState gains a carriers field; depth 0 publishes the working carriers themselves. The sync-free autoreset user guide documents the step-output lifetime contract, including which outputs remain per-step fresh (info, final_observation).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
TorchEnv.register_step_carrier(name, carrier): tasks register a device tensor they update in place duringupdate_state(for example a transition snapshot). Withpersistent_step_buffers(implied byautoreset_cuda_graph) the env copies it into the current publication-ring slot at every step boundary with pre-allocated,empty_likeslot buffers — so consumers hold the published zero-copy view inTorchEnvState.carriersforpublished_buffer_depthfurther steps instead of defending with per-stepdetach().clone().TorchEnvStategains acarriersfield; ring depth 0 publishes the working carriers themselves (zero-copy).reset(ring allocated), without the fixed-shape chain, on duplicate/empty names, and on non-tensor or wrong-device carriers.register_step_carrier; the step-output lifetime contract (what is ring-persistent vs per-step fresh) is now documented.Linked Work
mainRelated ADR context: no new structural decision — this completes the carrier contract #2605's ring started. ADR-0238's rejected state-update graph pre-registration already records why fixed carrier addresses are the capture precondition; this PR is the missing step-output side of that contract, not a revival of the rejected boundary.
Validation
make test-allpassed on the final local head before this PR was created or updatedCommands actually run (final head
810692b2, Linux, Python 3.13, CUDA available):Note:
tests/envs/test_multi_entity_consumer.pyinitially failed in the fresh worktree venv withModuleNotFoundError: mjbatch; that was a missingmujocoextra in the new venv, not a code issue — afteruv sync --all-extrasall four pass.Remote CI route:
main: not scheduled yet at PR creation; recording the current-head CI link and status here once the run exists.Impact
TorchEnv; noSimBackendcontract change)Artifacts
Checklist
Follow-up (out of scope here): #2627 compute-segment CUDA graph capture; per-slot
final_observationpersistence (currently documented as aliasing the working scratch — persisting it per ring slot would add per-step obs-width copies on the fixed-shape path, so it was left as a documented short-lived output).