Skip to content

feat(env): collapse autoreset-graph and validation config surface, default graph path on (#2617-#2621) - #2622

Merged
TATP-233 merged 11 commits into
mainfrom
dev/issue-2617-2621-autoreset-config-cleanup
Nov 19, 2026
Merged

TATP-233 merged 11 commits into
mainfrom
dev/issue-2617-2621-autoreset-config-cleanup

Conversation

@TATP-233

@TATP-233 TATP-233 commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

概述

集成分支 PR,一次性关闭 #2617、#2618、#2619、#2620、#2621 —— 五个相关联的 env 配置面 / base 公共合同 work item,按依赖结构在 dev/issue-2617-2621-autoreset-config-cleanup 上集成为一个可评审单元。

#2617 —— 收敛 autoreset CUDA graph 开关链(feat/issue-2617-collapse-autoreset-graph-switches)

  • env.autoreset_cuda_graph 成为唯一公共开关;设为 true 即隐含整条 fixed-shape 链(masked_autoreset、persistent_step_buffers、推迟有限性校验)。中间层开关从公共配置面删除(保留测试专用注入入口;已删除键以明确配置错误 fail closed)。
  • published_buffer_depth / autoreset_graph_warmup 降级为 TorchEnv 内部常量。
  • 四个 algo conf、任务样例 YAML、文档与 run_config.json 快照键同步清理(经真实 flashsac + g1_motion_tracking + mjwarp 训练对照:快照仅差三个被删键)。
  • 未改动:AutoresetGraph 捕获协议、ResetSelection 协商、eager fallback 路径、DP whole-cycle graph 逃生口。

#2618 —— 单一 validation strictness 概念(feat/issue-2618-validation-strictness)

  • 三处独立的"推迟有限性校验"开关(env 层 defer_finite_validation、NanGuardCfg.defer_validation、后端 mjwarp_defer_tensor_validation / mjwarp_validate_tensor_finite)收敛为一个 env 持有的概念:env.finite_validation_mode("strict" / "deferred",默认 "deferred")+ env.finite_validation_interval。
  • training 透传删除(training.finite_validation_interval、training.nan_guard.defer_validation);被删键以指明替代键的配置错误 fail closed;后端选项由 mode 派生。
  • scope decision 已记录在 issue 评论:env 层持有概念、training 透传删除、后端经现有 factory 协商路径派生。
  • 行为等价已在 MJWarp(RTX 4090)验证:strict 对 NaN action 仍在第 1 步立即抛错;deferred 仍在 interval=32 的轮询边界抛错。

#2619 —— 移除公共 reset 选择器的 slice 形式(feat/issue-2619-remove-reset-selector-slice)

  • resolve_reset_env_ids / ResetMaskBuffer.resolve / fill_rows_ / assign_rows_ 不再接受 slice,新增 TypeError 回归测试锁定;managers 与 motion-tracking terms 中的签名级联收窄(mypy 强制)。
  • 保留形式:None、int 行索引 tensor、ResetSelection。

#2620 —— TorchEnv.step 计时收敛为结构化 phase 计时器(feat/issue-2620-phase-timer)

  • 新增 unilab.base.phase_timer.PhaseTimer;torch_env.py / manager_based_rl_env.py 中约 15 处手工 perf_counter / os.times 簿记收敛为一行 with timer.phase(...) 声明。
  • info["timing"] 对外数据合同不变——键名、嵌套、backend 合并、reset 细分清除逻辑、键序均经 stub env 重构前后逐键对照验证一致。

#2621 —— env.autoreset_cuda_graph 默认开启(feat/issue-2621-autoreset-graph-default-on)

  • 单一开关默认值翻转为 true;显式 false 保留为文档标注的逃生口(调试、疑似 graph 问题时回退)。不做静默 eager 回退:不支持的组合(非 CUDA 设备、不接受 ResetSelection 的参与者、未声明 external_graph_capture 的后端)仍在启动时按名 fail closed。
  • 钉住 eager 生命周期的既有测试显式设 false;新增默认值开启、显式回退、fail-closed 三类断言。

平台 / 后端影响

仅通过配置默认值影响任务行为:CUDA + 声明 external_graph_capture 的后端上,fixed-shape autoreset 路径默认开启;需要 eager 选中行生命周期的运行须显式设置 ++env.autoreset_cuda_graph=false。masked 路径与 eager 路径的训练轨迹统计等价但不逐位相同(RNG 消耗顺序差异),该差异已在 #2621 中声明并被接受。

验证(最终 head 19f73aeb)

各子分支在各自 worktree 内集成前:

  • make check —— 五个子分支全部通过(ruff/mypy/pyright;一条与本草无关的 base/__init__.py 既有 pyright warning)。
  • focused tests(-m "not slow")—— 各子分支全绿,含每个 issue 新增/更新的回归测试。

集成分支最终 head(本 PR):

  • make check —— 通过。
  • make test-cov —— 3055 passed, 37 skipped, 380 deselected, 1 xfailed。
  • make test-benchmark-smoke —— module-mode 35/35,script-mode 36/36。
  • 说明:对修复前 head 的全量测试曾暴露 tests/envs/test_go2_superdex.py 2 个失败(packed-reset blocker 用例在 CPU 上被新默认值的 fail-closed 先行拦截),已在 19f73aeb 钉住 eager 生命周期修复,并在修复后 head 重跑全量全绿。

Closes #2617, closes #2618, closes #2619, closes #2620, closes #2621.

resolve_reset_env_ids / resolve_reset_mask / ResetMaskBuffer.resolve and
the fill_rows_ / assign_rows_ signatures no longer accept a Python slice;
a slice now raises TypeError with guidance to pass explicit row indices.
Manager and term reset signatures that feed these helpers are narrowed to
match, and the only in-repo test constructing a selector from a slice now
uses arange-sliced explicit ids.

The remaining public forms are None (all rows), a 1-D integer row-index
tensor, and the internal ResetSelection protocol; no deprecation alias is
kept (roadmap #2434 naming rules). Closes #2619.
…ructured phase timer

Collapse the per-phase perf_counter/os.times bookkeeping in TorchEnv.step,
TorchEnv._reset_done_envs, and ManagerBasedRlEnv.update_state into
declarative 'with timer.phase(...)' statements on the new PhaseTimer owner
in unilab.base. Wall/CPU paired sampling, phase accumulation across
non-contiguous segments, drain re-attribution (move_wall), the
env_step_other_cpu_ms residual, and reset/update_state detail gaps are
computed by the timer at publication time.

info["timing"] key names, nesting, semantics, and phase boundaries are
unchanged; the fake-clock tests now patch the phase_timer module. No
backend-side timing is touched.

Issue: #2620
… switch

env.autoreset_cuda_graph=true now implies masked autoreset, persistent
step buffers and deferred finite validation. The intermediate switches
(env.masked_autoreset, env.persistent_step_buffers,
env/training.defer_finite_validation), env.published_buffer_depth and
env.autoreset_graph_warmup leave the public config surface: the chain
switches are derived from the single switch, the ring depth and graph
warmup become TorchEnv-owned constants, and deleted keys fail closed
with a clear config error on the Hydra path. Tests inject intermediate
states through TorchEnv's documented soft-getattr protocol.

The graph capture protocol (AutoresetGraph), ResetSelection negotiation
and the eager fallback path are unchanged. Syncs the four algo confs,
the sync-free autoreset guide, the algorithm config reference tables and
the changelog; training routes only finite_validation_interval now.

Closes #2617
… dev/issue-2617-2621-autoreset-config-cleanup
Flip the single public fixed-shape autoreset switch to default-on
(issue #2621), matching the default-enabled precedent of the FlashSAC
DP whole-cycle CUDA graph. Task configs that do not mention the switch
now run the masked full-width reset, persistent step buffers, deferred
finite validation and — on CUDA — autoreset segment CUDA graph capture.

Unsupported combinations keep the existing fail-closed startup report
with the switch, participant or backend named (non-CUDA env device,
reset participants without ResetSelection support, backends without
the external_graph_capture capability); there is no silent eager
fallback. An explicit env.autoreset_cuda_graph=false remains as the
escape hatch for debugging or suspected graph-related problems.

Tests and tools that exercise the eager selected-row lifecycle on CPU
backends pin the switch off explicitly; the sync-free autoreset guide,
the training index card and the changelog document the new default and
the opt-out.

Closes #2621
…dation strictness concept

env.finite_validation_mode ("strict" or "deferred", default "deferred")
together with env.finite_validation_interval is now the single public
config point for finite (NaN/Inf) validation. It derives all three former
switches: the env's deferred finite validation, the NaN guard defer mode
(an attached guard follows the env via TorchEnv.set_nan_guard), and the
MJWarp backend options (strict keeps the backend's immediate per-call
operand check; deferred latches its fail-closed checks into device
verdicts published at the env poll).

The removed training.finite_validation_interval pass-through,
training.nan_guard.defer_validation and the backend switches
env.mjwarp_validate_tensor_finite / env.mjwarp_defer_tensor_validation
fail closed with a clear config error naming the replacement; env-level
deleted keys keep failing closed through config materialization. Judgment
semantics and the verdict carriers are unchanged.

Syncs the four algo confs (interval ships in the env block), the
sync-free autoreset guide, the training diagnostics pages, the algorithm
config reference tables and the changelog.

Closes #2618
…sue-2617-2621-autoreset-config-cleanup

# Conflicts:
#	docs/sphinx/source/changelog.md
#	docs/sphinx/source/en/2-user_guide/1-training/8-sync_free_autoreset.md
#	src/unilab/envs/manager_based_rl_env.py
…tests

The default-on autoreset_cuda_graph (#2621) fails closed on the CPU test
device before the superdex packed-reset blocker fires; these tests pin the
eager lifecycle explicitly so they keep covering the packed_reset error.
@TATP-233
TATP-233 requested a review from caozx1110 as a code owner November 19, 2026 12:29
@TATP-233

Copy link
Copy Markdown
Collaborator Author

Final-head validation (head 19f73aeb):

  • make check — passed (ruff format/check, mypy 0 issues, pyright 0 errors; one pre-existing base/__init__.py warning unrelated to this series).
  • make test-cov — 3055 passed, 37 skipped, 380 deselected, 1 xfailed.
  • make test-benchmark-smoke — module-mode 35/35, script-mode 36/36.

Note: an earlier full run on the pre-fix head surfaced 2 failures in tests/envs/test_go2_superdex.py (packed-reset blocker tests intercepted by the new default-on graph fail-closed on CPU); fixed in 19f73aeb by pinning the eager lifecycle in those tests, and the suite above re-ran green on the final head.

@TATP-233
TATP-233 merged commit 7d57345 into main Nov 19, 2026
20 of 22 checks passed
@TATP-233
TATP-233 deleted the dev/issue-2617-2621-autoreset-config-cleanup branch November 19, 2026 13:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant