diff --git a/docs/sphinx/source/changelog.md b/docs/sphinx/source/changelog.md index 582c33267..9851d7048 100644 --- a/docs/sphinx/source/changelog.md +++ b/docs/sphinx/source/changelog.md @@ -79,6 +79,70 @@ PyPI 版本变更与未发布变更;发布日期采用 PyPI 上传日期。完 ### Changed / 变更 +- Collapse the fixed-shape autoreset switch chain into the single public + switch `env.autoreset_cuda_graph`: enabling it now implies masked autoreset, + persistent step buffers and deferred finite validation. The intermediate + switches (`env.masked_autoreset`, `env.persistent_step_buffers`, + `env.defer_finite_validation` / `training.defer_finite_validation`), + `env.published_buffer_depth` and `env.autoreset_graph_warmup` are removed + from the public config surface (they are derived or internal constants); + supplying a deleted key fails closed with a clear config error. Behavior of + the graph capture protocol, `ResetSelection` negotiation and the eager + fallback path is unchanged + ([#2617](https://github.com/Motphys/UniLab/issues/2617))。 + 收敛 fixed-shape autoreset 开关链为单一公共开关 `env.autoreset_cuda_graph`: + 启用它即隐含 masked autoreset、持久 step buffer 与推迟有限值校验。中间层 + 开关(`env.masked_autoreset`、`env.persistent_step_buffers`、 + `env.defer_finite_validation` / `training.defer_finite_validation`)、 + `env.published_buffer_depth` 与 `env.autoreset_graph_warmup` 从公共配置面 + 删除(改为内部派生值或常量);传入已删除的键会以明确的配置错误 fail + closed。graph 捕获协议、`ResetSelection` 协商与 eager fallback 路径行为 + 不变([#2617](https://github.com/Motphys/UniLab/issues/2617))。 + +- Collapse the three deferred finite-validation switches into the single + env-owned validation strictness concept: `env.finite_validation_mode` + (`"strict"` or `"deferred"`, default `"deferred"`) together with + `env.finite_validation_interval`. The env's deferred finite validation, the + NaN guard defer mode (an attached guard follows the env), and the MJWarp + backend options are all derived from it (strict keeps the backend's + immediate per-call operand check; deferred latches its fail-closed checks + into device verdicts published at the env poll). The removed + `training.finite_validation_interval` pass-through, + `training.nan_guard.defer_validation` and the backend switches + `env.mjwarp_validate_tensor_finite` / `env.mjwarp_defer_tensor_validation` + fail closed with a clear config error naming the replacement. Judgment + semantics and the verdict carriers are unchanged + ([#2618](https://github.com/Motphys/UniLab/issues/2618))。 + 收敛三处"推迟有限性校验"开关为 env 持有的单一校验严格度概念: + `env.finite_validation_mode`(`"strict"` 或 `"deferred"`,默认 + `"deferred"`)与 `env.finite_validation_interval`。env 的推迟有限性校验、 + NaN guard 的 defer 模式(挂载的 guard 跟随 env)与 MJWarp 后端选项均由其 + 派生(strict 保持后端逐调用算子检查立即生效;deferred 将 fail-closed 检查 + 锁存为设备 verdict,由 env 在轮询边界发布)。删除的 + `training.finite_validation_interval` 透传、`training.nan_guard.defer_validation` + 与后端开关 `env.mjwarp_validate_tensor_finite` / + `env.mjwarp_defer_tensor_validation` 会以指明替代键的明确配置错误 fail + closed。判定语义与 verdict 载体不变 + ([#2618](https://github.com/Motphys/UniLab/issues/2618))。 +- `env.autoreset_cuda_graph` now defaults to `true`: task configs that do not + mention the switch run the fixed-shape autoreset path (masked full-width + reset, persistent step buffers, deferred finite validation, and — on CUDA — + autoreset segment CUDA graph capture). Unsupported combinations keep failing + closed at startup with the switch or backend named (a non-CUDA env device, + reset participants that do not accept a `ResetSelection`, or a backend + without the `external_graph_capture` capability); there is no silent eager + fallback. Set `env.autoreset_cuda_graph=false` explicitly to opt out — the + escape hatch for debugging or when a graph-related problem is suspected + ([#2621](https://github.com/Motphys/UniLab/issues/2621))。 + `env.autoreset_cuda_graph` 默认值翻转为 `true`:任务配置不写该开关时即运行 + fixed-shape autoreset 路径(masked 全宽 reset、持久 step buffer、推迟有限值 + 校验,CUDA 下捕获 autoreset 段 CUDA graph)。不支持的组合仍在启动时按名 fail + closed(非 CUDA env 设备、不接受 `ResetSelection` 的 reset 参与者、或未声明 + `external_graph_capture` 能力的后端);不做静默 eager 回退。显式设置 + `env.autoreset_cuda_graph=false` 即可退出——用于调试或疑似 graph 相关问题时的 + 回退逃生口 + ([#2621](https://github.com/Motphys/UniLab/issues/2621))。 + - High-traffic public interfaces — the Manager protocols in `unilab.managers.types`, `TorchEnv`, the reset-state transaction, the MDP observation/action/curriculum/ray terms, and the PPO/APPO/SAC/FlashSAC @@ -126,6 +190,24 @@ PyPI 版本变更与未发布变更;发布日期采用 PyPI 上传日期。完 路径。 ([#2610](https://github.com/Motphys/UniLab/issues/2610)). +### Removed / 移除 + +- The public reset selector no longer accepts a Python `slice`: the slice + branch is removed from `resolve_reset_env_ids` / `resolve_reset_mask` / + `ResetMaskBuffer.resolve` and from the `fill_rows_` / `assign_rows_` + signatures. Passing a slice now raises `TypeError`; pass explicit row + indices instead (e.g. `torch.arange(num_envs)[slice]`) or `None` for all + rows. The remaining forms are `None`, a 1-D integer row-index tensor, and + the internal `ResetSelection` protocol; no deprecation alias is kept + ([#2619](https://github.com/Motphys/UniLab/issues/2619)). + 公共 reset 选择器不再接受 Python `slice`:`resolve_reset_env_ids` / + `resolve_reset_mask` / `ResetMaskBuffer.resolve` 的 slice 分支与 + `fill_rows_` / `assign_rows_` 签名中的 slice 形式均已删除。传入 slice + 现在抛 `TypeError`;请改用显式行索引(如 `torch.arange(num_envs)[slice]`) + 或用 `None` 表示全部行。保留形式为 `None`、一维整数行索引张量与内部 + `ResetSelection` 协议;不留 deprecation 别名 + ([#2619](https://github.com/Motphys/UniLab/issues/2619))。 + ## 1.5.11 (2026-11-16) ### Added / 新增 diff --git a/docs/sphinx/source/en/2-user_guide/1-training/0-index.md b/docs/sphinx/source/en/2-user_guide/1-training/0-index.md index 7c6a248fd..b1df8a266 100644 --- a/docs/sphinx/source/en/2-user_guide/1-training/0-index.md +++ b/docs/sphinx/source/en/2-user_guide/1-training/0-index.md @@ -53,7 +53,7 @@ Reproduce and operate the scoped single-GPU G1 FlashSAC/MJWarp long soak. :::{grid-item-card} Sync-free autoreset :link: 8-sync_free_autoreset :link-type: doc -Opt-in fixed-shape autoreset, reset CUDA graph, and `ResetSelection` terms. +Default-on fixed-shape autoreset, reset CUDA graph, and `ResetSelection` terms. ::: :::: diff --git a/docs/sphinx/source/en/2-user_guide/1-training/8-sync_free_autoreset.md b/docs/sphinx/source/en/2-user_guide/1-training/8-sync_free_autoreset.md index 5f6a29c86..77ad993ad 100644 --- a/docs/sphinx/source/en/2-user_guide/1-training/8-sync_free_autoreset.md +++ b/docs/sphinx/source/en/2-user_guide/1-training/8-sync_free_autoreset.md @@ -5,9 +5,10 @@ mask on the host (`any()`), builds `env_ids` with `nonzero`, and launches reset kernels whose shape depends on the number of reset rows. Each of these steps synchronizes the host with the GPU. -An opt-in fixed-shape path removes these synchronizations, and on CUDA it can -replay the whole reset segment as one CUDA graph. Every switch defaults to -`False`, so tasks that do not opt in keep the selected-row behavior unchanged. +A fixed-shape path removes these synchronizations, and on CUDA it can +replay the whole reset segment as one CUDA graph. The path is controlled by a +single public switch, `env.autoreset_cuda_graph`, which defaults to `true`; +tasks that opt out with an explicit `false` keep the selected-row behavior. Implementation: `src/unilab/base/torch_env.py`, `src/unilab/base/reset_selection.py`, `src/unilab/base/autoreset_graph.py`, @@ -21,8 +22,6 @@ Implementation: `src/unilab/base/torch_env.py`, `ResetSelection` (see "Opting terms into `ResetSelection`" below). The G1 motion-tracking terms (`MotionCommand`, `JointPositionAction`, the motion body rewards and the anchor/body height terminations) already opt in. -- `training.defer_finite_validation=true`, the default in every shipped - algorithm config. - For the CUDA graph: a CUDA env device, and a backend that declares the `external_graph_capture` capability and implements `torch_graph_capture_scope`. MJWarp does both from `unisim-core` 2.7.2, the @@ -30,35 +29,48 @@ Implementation: `src/unilab/base/torch_env.py`, ## Configuration flags -All flags are fields of `ManagerBasedRlEnvCfg` (or of the base `EnvCfg` for the -backend flag). They live in the `env` section of the owner YAML. They are not -present in the shipped owner YAMLs, so set them with `++` from the command line: +`env.autoreset_cuda_graph` is the single public switch and defaults to +`true`. It implies the whole fixed-shape chain; the intermediate switches +(`masked_autoreset`, `persistent_step_buffers`), the publication ring depth and +the graph warmup are internal — derived from this switch or owned as constants +by `TorchEnv` — and are no longer configuration keys. Supplying a deleted key +fails closed with a config error. The fixed-shape chain also requires the +deferred finite validation mode: `finite_validation_mode='deferred'` is the +default, and `autoreset_cuda_graph=true` with `finite_validation_mode='strict'` +fails closed at env construction. Set the switch to `false` explicitly to opt +out; that escape hatch is the fallback for debugging or when a graph-related +problem is suspected (the run then uses the selected-row eager path). ```bash -uv run train --algo flashsac --task g1_motion_tracking --sim mjwarp \ - ++env.masked_autoreset=true \ - ++env.persistent_step_buffers=true \ - ++env.autoreset_cuda_graph=true \ - ++env.mjwarp_defer_tensor_validation=true +uv run train --algo flashsac --task g1_motion_tracking --sim mjwarp ``` | Flag | Default | Effect | | --- | --- | --- | -| `masked_autoreset` | `False` | Runs the autoreset on every step as a full-width `reset(ResetSelection)` under the device done mask, with no `any()` gate and no `nonzero`. Requires `defer_finite_validation`. | -| `persistent_step_buffers` | `False` | Allocates the observation, reward, terminated and truncated carriers once and updates them with `copy_`. Consumers receive one of `published_buffer_depth` persistent publication slots. | -| `published_buffer_depth` | `2` | Size of the publication ring. A published state stays valid for `published_buffer_depth - 1` further steps. `0` publishes the working carriers themselves, which the next step overwrites. | -| `autoreset_cuda_graph` | `False` | Captures the masked autoreset segment, including the backend reset, into one `torch.cuda.graph` and replays it afterwards. Requires `masked_autoreset`, `persistent_step_buffers`, a CUDA device and a backend with external graph capture. | -| `autoreset_graph_warmup` | `2` | Eager executions before capture, so that lazily allocated buffers exist outside the graph. | +| `autoreset_cuda_graph` | `True` | Runs the autoreset on every step as a full-width `reset(ResetSelection)` under the device done mask (no `any()` gate, no `nonzero`), keeps the observation, reward, terminated and truncated carriers persistent, defers every fail-closed finite check to the poll boundary, and — on CUDA — captures the masked autoreset segment, including the backend reset, into one `torch.cuda.graph` and replays it afterwards. Requires a CUDA device, a backend with external graph capture, and `finite_validation_mode='deferred'`. | | `autoreset_graph_capture_mode` | `"thread_local"` | `torch.cuda.graph` capture error mode: `"global"`, `"thread_local"` or `"relaxed"`. | -| `mjwarp_defer_tensor_validation` | `False` | Latches MJWarp's fail-closed operand and reset-mask checks into device verdicts instead of reading each one on the host. The env publishes them at its deferred-validation poll. Inside a graph capture MJWarp always defers. The flag also removes the host reads from the eager steps outside the captured segment. UniLab forwards it only when it is `True`. | -Invalid combinations fail when the config is validated: for example, -`autoreset_cuda_graph` without `masked_autoreset` and `persistent_step_buffers`, -or `masked_autoreset` without `defer_finite_validation`. +The MJWarp backend options follow the env's single validation strictness +(`env.finite_validation_mode`, issue #2618): deferred mode latches MJWarp's +fail-closed operand and reset-mask checks into device verdicts (the removed +`mjwarp_defer_tensor_validation` switch) that the env publishes at its +deferred-validation poll, and also removes the host reads from the eager steps +outside the captured segment; strict mode keeps MJWarp's per-call operand check +immediate (`mjwarp_validate_tensor_finite`). Inside a graph capture MJWarp +always defers. + +A published state stays valid for one further step (the ring holds two +persistent slots, which the off-policy double-buffer contract needs); the +capture happens after two eager warmup executions so that lazily allocated +buffers exist outside the graph. + +Invalid combinations fail when the environment is constructed: for example, +`autoreset_cuda_graph` on a non-CUDA device, or a backend without the +`external_graph_capture` capability. ## Behavior notes -**RNG draw order.** With `masked_autoreset` the reset lifecycle runs on every +**RNG draw order.** On the fixed-shape path the reset lifecycle runs on every step; an all-false mask leaves the state unchanged. Full-width samplers, such as the `MotionCommand` resample or the reset-event interval times, therefore draw from the Manager RNG on steps where no row is done. With the same seed, @@ -73,15 +85,14 @@ on done rows. latched on the device. This covers finite checks, `TorchEnv.require` verdicts, reset-mask containment and MJWarp/packed-reset verdicts. `poll_finite_validation` reads all of them together every -`training.finite_validation_interval` steps (32 in the shipped configs). A +`env.finite_validation_interval` steps (32 in the shipped configs). A failure is therefore raised at the next poll, up to `interval - 1` steps after the faulty step. Packed-reset failures name the poll step (`first detected at poll boundary, step N`). To localize a fault, rerun with -`training.finite_validation_interval=1`; the masked path still requires -`training.defer_finite_validation=true`. +`env.finite_validation_interval=1` so every step is a poll boundary. -**Stale diagnostic counters.** In masked mode the env has no host-side reset row -count or per-phase split. Consequences: +**Stale diagnostic counters.** On the fixed-shape path the env has no +host-side reset row count or per-phase split. Consequences: - The per-reset detail keys in `info["timing"]` (`RESET_DONE_DETAIL_TIMING_KEYS` in `src/unilab/base/backend_timing.py`, for @@ -120,7 +131,7 @@ support and handling the selection as a masked full-width write. calls a factory `values(n)` that returns `n` rows (`num_envs` rows under a selection). - Both accept row ids, `None`, a slice or a selection. + Both accept row ids, `None`, or a selection. - Do not read the mask on the host (`bool(mask.any())`, `.item()`, `nonzero`) inside `reset`; that reintroduces the synchronization. @@ -157,7 +168,7 @@ Some managers impose extra rules: ### Debugging `reset_selection_blockers` -With `masked_autoreset`, `init_state` calls +With the fixed-shape path, `init_state` calls `ManagerBasedRlEnv.reset_selection_blockers()`. It collects one entry per unsupported participant from every manager and raises: @@ -169,13 +180,13 @@ ResetSelection; unsupported: EventManager: (); ... Each entry has the form `: ()`. A manager that cannot take a selection for a structural reason reports a sentence instead. One example is `TerminationManager: episode counts are read on the -host unless defer_finite_validation is enabled`. +host unless finite_validation_mode='deferred'`. To list blockers without starting a training run, construct the env with the flag off and call the method: ```python -env.reset_selection_blockers() # [] when the task can enable masked_autoreset +env.reset_selection_blockers() # [] when the task can enable the fixed-shape path ``` Then convert, replace or remove each named term. Reset-randomization events are diff --git a/docs/sphinx/source/en/2-user_guide/2-algorithms/1-ppo.md b/docs/sphinx/source/en/2-user_guide/2-algorithms/1-ppo.md index a66e87e33..ff5ba1574 100644 --- a/docs/sphinx/source/en/2-user_guide/2-algorithms/1-ppo.md +++ b/docs/sphinx/source/en/2-user_guide/2-algorithms/1-ppo.md @@ -147,8 +147,7 @@ uv run train --algo ppo --task go2_joystick_flat --sim mujoco \ | `training.tf32_matmul` | str | ieee | Learner FP32 matmul backend; `tf32` opts into tensor cores with automatic IEEE fallback | | `training.nan_guard.enabled` | bool | true | Detect non-finite tensors and dump diagnostics | | `training.nan_guard.check_interval` | int | 32 | Env steps between NaN checks | -| `training.defer_finite_validation` | bool | true | Batch finite checks into a periodic cycle | -| `training.finite_validation_interval` | int | 32 | Steps per batched finite-check cycle | +| `env.finite_validation_interval` | int | 32 | Steps per batched finite-check poll cycle (env-owned validation strictness) | | `training.collector_tensor_device` | str | auto | Device of collector-side tensors; `auto` follows the learner | | `training.logger` | str | tensorboard | Scalar logging backend (`tensorboard` or `wandb`) | | `training.num_timesteps` | int / null | null | Global sample budget; overrides the derived iteration count when set | diff --git a/docs/sphinx/source/en/2-user_guide/2-algorithms/2-appo.md b/docs/sphinx/source/en/2-user_guide/2-algorithms/2-appo.md index b7165f812..910a52a47 100644 --- a/docs/sphinx/source/en/2-user_guide/2-algorithms/2-appo.md +++ b/docs/sphinx/source/en/2-user_guide/2-algorithms/2-appo.md @@ -138,8 +138,7 @@ uv run train --algo appo --task go2_joystick_flat --sim mujoco \ | `training.tf32_matmul` | str | ieee | Learner FP32 matmul backend; `tf32` opts into tensor cores with automatic IEEE fallback | | `training.nan_guard.enabled` | bool | true | Detect non-finite tensors and dump diagnostics | | `training.nan_guard.check_interval` | int | 32 | Env steps between NaN checks | -| `training.defer_finite_validation` | bool | true | Batch finite checks into a periodic cycle | -| `training.finite_validation_interval` | int | 32 | Steps per batched finite-check cycle | +| `env.finite_validation_interval` | int | 32 | Steps per batched finite-check poll cycle (env-owned validation strictness) | | `training.collector_tensor_device` | str | auto | Device of collector-side tensors; `auto` follows the learner | | `training.logger` | str | tensorboard | Scalar logging backend (`tensorboard` or `wandb`) | diff --git a/docs/sphinx/source/en/2-user_guide/2-algorithms/3-sac.md b/docs/sphinx/source/en/2-user_guide/2-algorithms/3-sac.md index dcb796daf..5b05f5aac 100644 --- a/docs/sphinx/source/en/2-user_guide/2-algorithms/3-sac.md +++ b/docs/sphinx/source/en/2-user_guide/2-algorithms/3-sac.md @@ -118,8 +118,7 @@ uv run train --algo sac --task g1_walk_flat --sim mujoco \ | `training.dp_gradient_wire_dtype` | str | float32 | Gradient wire dtype for multi-GPU averaging; `bfloat16` compresses | | `training.nan_guard.enabled` | bool | true | Detect non-finite tensors and dump diagnostics | | `training.nan_guard.check_interval` | int | 32 | Env steps between NaN checks | -| `training.defer_finite_validation` | bool | true | Batch finite checks into a periodic cycle | -| `training.finite_validation_interval` | int | 32 | Steps per batched finite-check cycle | +| `env.finite_validation_interval` | int | 32 | Steps per batched finite-check poll cycle (env-owned validation strictness) | | `training.collector_tensor_device` | str | auto | Device of collector-side tensors; `auto` follows the learner | | `training.env_steps_per_sync` | int | 1 | Env steps per collection sync | | `training.inference_slot_capacity` | int | 1 | In-flight learner-inference requests; 1 keeps inference serialized | diff --git a/docs/sphinx/source/en/2-user_guide/2-algorithms/4-flash_sac.md b/docs/sphinx/source/en/2-user_guide/2-algorithms/4-flash_sac.md index 185683de3..a407087ce 100644 --- a/docs/sphinx/source/en/2-user_guide/2-algorithms/4-flash_sac.md +++ b/docs/sphinx/source/en/2-user_guide/2-algorithms/4-flash_sac.md @@ -151,8 +151,7 @@ uv run train --algo flashsac --task g1_walk_flat --sim mujoco \ | `training.dp_gradient_wire_dtype` | str | float32 | Gradient wire dtype for multi-GPU averaging; `bfloat16` compresses | | `training.nan_guard.enabled` | bool | true | Detect non-finite tensors and dump diagnostics | | `training.nan_guard.check_interval` | int | 32 | Env steps between NaN checks | -| `training.defer_finite_validation` | bool | true | Batch finite checks into a periodic cycle | -| `training.finite_validation_interval` | int | 32 | Steps per batched finite-check cycle | +| `env.finite_validation_interval` | int | 32 | Steps per batched finite-check poll cycle (env-owned validation strictness) | | `training.collector_tensor_device` | str | auto | Device of collector-side tensors; `auto` follows the learner | | `training.env_steps_per_sync` | int | 1 | Env steps per collection sync | | `training.inference_slot_capacity` | int | 1 | In-flight learner-inference requests; 1 keeps inference serialized | diff --git a/docs/sphinx/source/en/6-how_to/3-training_diagnostics.md b/docs/sphinx/source/en/6-how_to/3-training_diagnostics.md index 0a5888b1b..6ca052f36 100644 --- a/docs/sphinx/source/en/6-how_to/3-training_diagnostics.md +++ b/docs/sphinx/source/en/6-how_to/3-training_diagnostics.md @@ -22,7 +22,6 @@ on-device and writes one state dump at an explicit host boundary. The chain: buffer_size: 100 max_envs_to_dump: 5 output_dir: null # null -> /tmp/unilab/nan_dumps - defer_validation: true check_interval: 32 capture_interval: 32 ``` @@ -30,7 +29,9 @@ on-device and writes one state dump at an explicit host boundary. The chain: Fields are defined by `NanGuardCfg` in `src/unilab/training/tensor_diagnostics.py`. `capture_interval` controls how often a physics-state snapshot is retained (0 disables snapshots); - `check_interval` is the poll period in deferred mode. + `check_interval` is the poll period in deferred mode. The defer mode itself + is not a guard field: it follows the env's single validation strictness + (`env.finite_validation_mode`, issue #2618) when the guard is attached. 2. The training entrypoint attaches the guard to the env (`apply_env_nan_guard` in `src/unilab/training/run.py`), and `TorchEnv.step` calls `NanGuard.check(...)` each step. On detection, diff --git a/docs/sphinx/source/zh_CN/2-user_guide/2-algorithms/1-ppo.md b/docs/sphinx/source/zh_CN/2-user_guide/2-algorithms/1-ppo.md index 272d9c228..14d4c1f7e 100644 --- a/docs/sphinx/source/zh_CN/2-user_guide/2-algorithms/1-ppo.md +++ b/docs/sphinx/source/zh_CN/2-user_guide/2-algorithms/1-ppo.md @@ -139,8 +139,7 @@ uv run train --algo ppo --task go2_joystick_flat --sim mujoco \ | `training.tf32_matmul` | str | ieee | learner FP32 矩阵乘后端;`tf32` 启用 tensor core 并自动回退 IEEE | | `training.nan_guard.enabled` | bool | true | 检测非有限张量并转储诊断信息 | | `training.nan_guard.check_interval` | int | 32 | NaN 检查之间的 env step 数 | -| `training.defer_finite_validation` | bool | true | 将有限性检查合并为周期性批次 | -| `training.finite_validation_interval` | int | 32 | 每个批量有限性检查周期的步数 | +| `env.finite_validation_interval` | int | 32 | 每个批量有限性检查轮询周期的步数(env 持有的校验严格度) | | `training.collector_tensor_device` | str | auto | collector 侧张量设备;`auto` 跟随 learner | | `training.logger` | str | tensorboard | 标量日志后端(`tensorboard` 或 `wandb`) | | `training.num_timesteps` | int / null | null | 全局样本预算;设置后覆盖推导出的迭代数 | diff --git a/docs/sphinx/source/zh_CN/2-user_guide/2-algorithms/2-appo.md b/docs/sphinx/source/zh_CN/2-user_guide/2-algorithms/2-appo.md index a04d529dd..f9f8f3959 100644 --- a/docs/sphinx/source/zh_CN/2-user_guide/2-algorithms/2-appo.md +++ b/docs/sphinx/source/zh_CN/2-user_guide/2-algorithms/2-appo.md @@ -136,8 +136,7 @@ uv run train --algo appo --task go2_joystick_flat --sim mujoco \ | `training.tf32_matmul` | str | ieee | learner FP32 矩阵乘后端;`tf32` 启用 tensor core 并自动回退 IEEE | | `training.nan_guard.enabled` | bool | true | 检测非有限张量并转储诊断信息 | | `training.nan_guard.check_interval` | int | 32 | NaN 检查之间的 env step 数 | -| `training.defer_finite_validation` | bool | true | 将有限性检查合并为周期性批次 | -| `training.finite_validation_interval` | int | 32 | 每个批量有限性检查周期的步数 | +| `env.finite_validation_interval` | int | 32 | 每个批量有限性检查轮询周期的步数(env 持有的校验严格度) | | `training.collector_tensor_device` | str | auto | collector 侧张量设备;`auto` 跟随 learner | | `training.logger` | str | tensorboard | 标量日志后端(`tensorboard` 或 `wandb`) | diff --git a/docs/sphinx/source/zh_CN/2-user_guide/2-algorithms/3-sac.md b/docs/sphinx/source/zh_CN/2-user_guide/2-algorithms/3-sac.md index d84e0d179..0f7050cf9 100644 --- a/docs/sphinx/source/zh_CN/2-user_guide/2-algorithms/3-sac.md +++ b/docs/sphinx/source/zh_CN/2-user_guide/2-algorithms/3-sac.md @@ -162,8 +162,7 @@ uv run train --algo sac --task g1_walk_flat --sim mujoco \ | `training.dp_gradient_wire_dtype` | str | float32 | 多卡梯度平均的传输 dtype;`bfloat16` 会压缩 | | `training.nan_guard.enabled` | bool | true | 检测非有限张量并转储诊断信息 | | `training.nan_guard.check_interval` | int | 32 | NaN 检查之间的 env step 数 | -| `training.defer_finite_validation` | bool | true | 将有限性检查合并为周期性批次 | -| `training.finite_validation_interval` | int | 32 | 每个批量有限性检查周期的步数 | +| `env.finite_validation_interval` | int | 32 | 每个批量有限性检查轮询周期的步数(env 持有的校验严格度) | | `training.collector_tensor_device` | str | auto | collector 侧张量设备;`auto` 跟随 learner | | `training.env_steps_per_sync` | int | 1 | 每次采集同步的 env step 数 | | `training.inference_slot_capacity` | int | 1 | 在途 learner 推理请求数;1 表示严格串行 | diff --git a/docs/sphinx/source/zh_CN/2-user_guide/2-algorithms/4-flash_sac.md b/docs/sphinx/source/zh_CN/2-user_guide/2-algorithms/4-flash_sac.md index 0fba8602f..0ec1e2519 100644 --- a/docs/sphinx/source/zh_CN/2-user_guide/2-algorithms/4-flash_sac.md +++ b/docs/sphinx/source/zh_CN/2-user_guide/2-algorithms/4-flash_sac.md @@ -155,8 +155,7 @@ uv run train --algo flashsac --task g1_walk_flat --sim mujoco \ | `training.dp_gradient_wire_dtype` | str | float32 | 多卡梯度平均的传输 dtype;`bfloat16` 会压缩 | | `training.nan_guard.enabled` | bool | true | 检测非有限张量并转储诊断信息 | | `training.nan_guard.check_interval` | int | 32 | NaN 检查之间的 env step 数 | -| `training.defer_finite_validation` | bool | true | 将有限性检查合并为周期性批次 | -| `training.finite_validation_interval` | int | 32 | 每个批量有限性检查周期的步数 | +| `env.finite_validation_interval` | int | 32 | 每个批量有限性检查轮询周期的步数(env 持有的校验严格度) | | `training.collector_tensor_device` | str | auto | collector 侧张量设备;`auto` 跟随 learner | | `training.env_steps_per_sync` | int | 1 | 每次采集同步的 env step 数 | | `training.inference_slot_capacity` | int | 1 | 在途 learner 推理请求数;1 表示严格串行 | diff --git a/docs/sphinx/source/zh_CN/6-how_to/3-training_diagnostics.md b/docs/sphinx/source/zh_CN/6-how_to/3-training_diagnostics.md index 3b5bb8eac..ce96e4351 100644 --- a/docs/sphinx/source/zh_CN/6-how_to/3-training_diagnostics.md +++ b/docs/sphinx/source/zh_CN/6-how_to/3-training_diagnostics.md @@ -20,14 +20,15 @@ NaN guard 在设备侧检查观测、reward 与控制量中的非有限行,并 buffer_size: 100 max_envs_to_dump: 5 output_dir: null # null -> /tmp/unilab/nan_dumps - defer_validation: true check_interval: 32 capture_interval: 32 ``` 字段由 `src/unilab/training/tensor_diagnostics.py` 的 `NanGuardCfg` 定义。 `capture_interval` 控制物理状态快照的保留频率(0 表示不快照); - `check_interval` 是 deferred 模式下的轮询周期。 + `check_interval` 是 deferred 模式下的轮询周期。defer 模式本身不是 guard + 字段:guard 挂载时跟随 env 的单一校验严格度 + (`env.finite_validation_mode`,issue #2618)。 2. 训练入口把 guard 挂到 env 上(`src/unilab/training/run.py` 的 `apply_env_nan_guard`),`TorchEnv.step` 每步调用 `NanGuard.check(...)`。 检出时 `NanGuard.dump(...)` 写出 `nan_dump_<时间戳>_step.npz` diff --git a/scripts/tools/capture_mujoco_drift_baseline.py b/scripts/tools/capture_mujoco_drift_baseline.py index 659edb2ee..131856f9a 100644 --- a/scripts/tools/capture_mujoco_drift_baseline.py +++ b/scripts/tools/capture_mujoco_drift_baseline.py @@ -162,6 +162,9 @@ def _build_env(task_path: str, args: argparse.Namespace): { "seed": args.seed, "cpu_ids": list(args.cpu_ids), + # The baseline captures the eager selected-row backend behavior; + # opt out of the default-on autoreset graph path. + "autoreset_cuda_graph": False, } ) env = registry.make( diff --git a/scripts/visualize_task_env.py b/scripts/visualize_task_env.py index 56c53d985..40c908a64 100644 --- a/scripts/visualize_task_env.py +++ b/scripts/visualize_task_env.py @@ -250,7 +250,9 @@ def _build_env_cfg_override(task_name: str) -> dict[str, Any]: f"Task '{task_name}' is not registered. " f"Available: {sorted(registry.list_registered_envs())}" ) - return {} + # Visualization drives the eager selected-row path; opt out of the + # default-on autoreset graph path. + return {"autoreset_cuda_graph": False} def _print_backend_scene(env) -> None: diff --git a/src/unilab/base/backend_factory.py b/src/unilab/base/backend_factory.py index 8be30b825..14296d3c8 100644 --- a/src/unilab/base/backend_factory.py +++ b/src/unilab/base/backend_factory.py @@ -267,11 +267,28 @@ def env_backend_kwargs(cfg: "EnvCfg") -> dict[str, Any]: if cfg.genesis_device_id is not None: result["genesis_device_id"] = cfg.genesis_device_id result["newton_use_cuda_graph"] = cfg.newton_use_cuda_graph - # Always explicit: never fall back to UniSim's blocking MJWarp default. - # Other backends ignore this option. - result["mjwarp_validate_tensor_finite"] = cfg.mjwarp_validate_tensor_finite - if cfg.mjwarp_defer_tensor_validation: + # MJWarp finite validation derives from the env's single validation + # strictness (``finite_validation_mode``); the removed ``mjwarp_*`` + # switches are no longer owner config. Strict mode keeps the backend's + # per-call operand check immediate; deferred mode disables that duplicate + # check and latches the backend's fail-closed checks into device verdicts + # that the env publishes at its poll boundary. Other backends ignore + # these options. + for removed in ("mjwarp_validate_tensor_finite", "mjwarp_defer_tensor_validation"): + if hasattr(cfg, removed): + raise ValueError( + f"{removed} was removed from the public config surface: MJWarp finite " + "validation derives from env.finite_validation_mode " + "('strict' -> immediate backend checks, 'deferred' -> device verdicts)" + ) + mode = getattr(cfg, "finite_validation_mode", "deferred") + if mode == "strict": + result["mjwarp_validate_tensor_finite"] = True + elif mode == "deferred": + result["mjwarp_validate_tensor_finite"] = False result["mjwarp_defer_tensor_validation"] = True + else: + raise ValueError(f"finite_validation_mode must be 'strict' or 'deferred', got {mode!r}") return result diff --git a/src/unilab/base/base.py b/src/unilab/base/base.py index 7eb144e75..bf9f779ce 100644 --- a/src/unilab/base/base.py +++ b/src/unilab/base/base.py @@ -73,19 +73,6 @@ class EnvCfg: # device profile never relies on an implicit backend-wide allocation size. mjwarp_nconmax: Optional[int] = None mjwarp_njmax: Optional[int] = None - # MJWarp's own NaN/Inf check on hot-path tensor operands (step ctrl, device - # pushes, wrenches, velocity deltas, generalized forces) blocks the host on - # every call. TorchEnv already validates the final control (eagerly or - # through deferred finite validation), so UniLab always passes this value - # explicitly and disables the duplicate backend check by default. Set it to - # True only to debug a non-finite operand at the exact backend call. - mjwarp_validate_tensor_finite: bool = False - # Latch MJWarp's fail-closed operand/bounds checks into device verdicts - # instead of reading them on the host; TorchEnv publishes them with its - # deferred finite validation. Required for a host-sync-free step when - # mjwarp_validate_tensor_finite is True. Forwarded only when True, so - # UniSim releases without the option keep working. - mjwarp_defer_tensor_validation: bool = False # ``newton`` owns MuJoCo-Warp 3.11 capacity and requires explicit CUDA # placement; the process binder supplies the rank-local device when unset. newton_device: Optional[str] = None @@ -186,16 +173,6 @@ def validate(self): isinstance(value, bool) or not isinstance(value, int) or value <= 0 ): raise ValueError(f"{name} must be a positive integer or None, got {value!r}") - if not isinstance(self.mjwarp_validate_tensor_finite, bool): - raise ValueError( - "mjwarp_validate_tensor_finite must be bool, " - f"got {self.mjwarp_validate_tensor_finite!r}" - ) - if not isinstance(self.mjwarp_defer_tensor_validation, bool): - raise ValueError( - "mjwarp_defer_tensor_validation must be bool, " - f"got {self.mjwarp_defer_tensor_validation!r}" - ) if self.newton_device is not None and ( not isinstance(self.newton_device, str) or not self.newton_device.strip() ): diff --git a/src/unilab/base/phase_timer.py b/src/unilab/base/phase_timer.py new file mode 100644 index 000000000..57c7c1dc9 --- /dev/null +++ b/src/unilab/base/phase_timer.py @@ -0,0 +1,186 @@ +"""Structured per-phase wall/CPU timing collector for one host lifecycle. + +`PhaseTimer` replaces hand-written ``perf_counter`` / ``os.times`` +bookkeeping in hot environment paths (one ``TorchEnv.step``, one +``_reset_done_envs``, one ``update_state``). Each phase is declared with a +single ``with timer.phase(name):`` statement; the paired wall and process +CPU samples are taken by the timer itself, and derived quantities (elapsed +total, unattributed CPU residual, wall re-attribution between phases) are +computed from the recorded samples at publication time. + +The collector is host-only bookkeeping: it performs no device work and never +reads environment state, so it stays outside CUDA-graph-captured segments. +""" + +from __future__ import annotations + +import os +import time +from collections.abc import Iterator, MutableMapping +from contextlib import contextmanager +from dataclasses import dataclass +from typing import Any + + +def _cpu_time() -> float: + """Process CPU time (user + system) in seconds, for paired CPU samples.""" + process = os.times() + return process.user + process.system + + +@dataclass +class _PhaseSample: + """Accumulated milliseconds for one named phase.""" + + wall_ms: float = 0.0 + cpu_ms: float | None = None + + +class PhaseTimer: + """Collect paired wall/CPU timings for the phases of one host lifecycle. + + A timer instance covers exactly one lifecycle invocation (for example + one ``TorchEnv.step``). Construction captures the start clocks; each + ``phase`` context then accumulates its wall window, and — when CPU + collection is enabled — its process CPU window. Re-entering the same + phase name accumulates into the same sample, so a phase split across + non-contiguous segments (for example a backend step plus its later + done-gate host read) keeps one declared name. + + Derived values are computed on demand from the recorded samples: + :meth:`total_wall_ms` for the whole lifecycle, :meth:`residual_cpu_ms` + for the CPU time not attributed to any CPU-measured phase, and + :meth:`move_wall` for re-attributing a measured wall window between two + phases (for example draining an asynchronous backend step out of the + update-state phase). :meth:`publish` writes the recorded phases into a + timing mapping under ``"_ms"`` / ``"_cpu_ms"`` keys. + """ + + def __init__(self, *, cpu: bool = True) -> None: + """Start one lifecycle timing collection. + + Args: + cpu: Whether phases pair wall windows with process CPU windows + by default, and whether the CPU residual is computed against + the construction-time CPU clock. Wall-only collectors (for + example reset bookkeeping) pass ``cpu=False`` to skip every + ``os.times`` sample. + """ + self._cpu_enabled = cpu + self._wall_start = time.perf_counter() + self._cpu_start = _cpu_time() if cpu else None + self._phases: dict[str, _PhaseSample] = {} + self._marks: dict[str, float] = {} + + @contextmanager + def phase(self, name: str, *, cpu: bool | None = None) -> Iterator[None]: + """Accumulate one execution of the named phase into its sample. + + Args: + name: Phase name. Re-entering an existing name accumulates into + its sample, preserving one name for logically contiguous + phases that the code splits into non-adjacent segments. + cpu: Pair the wall window with a process CPU window. ``None`` + (default) follows the timer-wide ``cpu`` construction flag. + + Yields: + Nothing; the wrapped block is the measured phase body. + """ + if cpu is None: + cpu = self._cpu_enabled + if cpu and not self._cpu_enabled: + raise ValueError(f"phase {name!r} requested CPU time on a wall-only timer") + wall_started = time.perf_counter() + cpu_started = _cpu_time() if cpu else None + try: + yield + finally: + self._add( + name, + wall_ms=(time.perf_counter() - wall_started) * 1000.0, + cpu_ms=None if cpu_started is None else (_cpu_time() - cpu_started) * 1000.0, + ) + + def _add(self, name: str, *, wall_ms: float, cpu_ms: float | None) -> None: + sample = self._phases.get(name) + if sample is None: + sample = _PhaseSample() + self._phases[name] = sample + sample.wall_ms += wall_ms + if cpu_ms is not None: + sample.cpu_ms = cpu_ms if sample.cpu_ms is None else sample.cpu_ms + cpu_ms + + def mark(self, name: str) -> None: + """Record a wall-clock checkpoint for later :meth:`span_ms` reads. + + Marks cover windows that are not phase bodies (for example a span + that must include context-manager entry overhead). + """ + self._marks[name] = time.perf_counter() + + def span_ms(self, start: str, *, end: str | None = None) -> float: + """Return the wall milliseconds between two checkpoints. + + Args: + start: The mark recorded with :meth:`mark`. + end: Optional end mark; ``None`` (default) reads the clock now. + """ + wall_start = self._marks[start] + wall_end = time.perf_counter() if end is None else self._marks[end] + return (wall_end - wall_start) * 1000.0 + + def wall_ms(self, name: str) -> float: + """Return the accumulated wall milliseconds of one phase.""" + return self._phases[name].wall_ms + + def move_wall(self, source: str, target: str, wall_ms: float) -> None: + """Re-attribute a measured wall window from one phase to another. + + The source phase is debited (floored at zero); the target phase is + credited with the full window. Used for windows measured inside one + phase that semantically belong to another, such as an update-state + host read that drains an asynchronous backend step. + """ + source_sample = self._phases[source] + source_sample.wall_ms = max(source_sample.wall_ms - wall_ms, 0.0) + self._add(target, wall_ms=wall_ms, cpu_ms=None) + + def total_wall_ms(self) -> float: + """Return the wall milliseconds since timer construction.""" + return (time.perf_counter() - self._wall_start) * 1000.0 + + def total_cpu_ms(self) -> float: + """Return the process CPU milliseconds since timer construction.""" + if self._cpu_start is None: + raise ValueError("wall-only timer has no CPU clock") + return (_cpu_time() - self._cpu_start) * 1000.0 + + def residual_cpu_ms(self) -> float: + """Return the CPU milliseconds not attributed to any CPU phase. + + Computed as total CPU time minus the sum of every CPU-measured + phase, floored at zero. + """ + attributed = sum(sample.cpu_ms for sample in self._phases.values() if sample.cpu_ms) + return max(self.total_cpu_ms() - attributed, 0.0) + + def attributed_wall_ms(self) -> float: + """Return the sum of every recorded phase wall window.""" + return sum(sample.wall_ms for sample in self._phases.values()) + + def publish(self, timing: MutableMapping[str, Any]) -> None: + """Write every recorded phase into a timing mapping. + + Each phase ``name`` publishes ``"_ms"`` for its wall window + and, when CPU was measured, ``"_cpu_ms"`` for its CPU window. + Phases publish in declaration order. + """ + for name, sample in self._phases.items(): + timing[f"{name}_ms"] = sample.wall_ms + if sample.cpu_ms is not None: + timing[f"{name}_cpu_ms"] = sample.cpu_ms + + +__all__ = [ + "PhaseTimer", +] diff --git a/src/unilab/base/reset_selection.py b/src/unilab/base/reset_selection.py index 6829853ff..9f233192e 100644 --- a/src/unilab/base/reset_selection.py +++ b/src/unilab/base/reset_selection.py @@ -136,7 +136,7 @@ def _describe(participant: object) -> str: def fill_rows_( tensor: torch.Tensor, - rows: torch.Tensor | slice | ResetSelection | None, + rows: torch.Tensor | ResetSelection | None, value: float | int | bool, *, row_dim: int = 0, @@ -151,7 +151,7 @@ def fill_rows_( def assign_rows_( tensor: torch.Tensor, - rows: torch.Tensor | slice | ResetSelection | None, + rows: torch.Tensor | ResetSelection | None, values: torch.Tensor | Callable[[int], torch.Tensor], *, row_dim: int = 0, @@ -169,12 +169,7 @@ def assign_rows_( return index: tuple[Any, ...] = (slice(None),) * row_dim + (slice(None) if rows is None else rows,) if callable(values): - if rows is None: - count = tensor.shape[row_dim] - elif isinstance(rows, slice): - count = len(range(*rows.indices(tensor.shape[row_dim]))) - else: - count = int(rows.numel()) + count = tensor.shape[row_dim] if rows is None else int(rows.numel()) tensor[index] = values(count) return tensor[index] = values[index] diff --git a/src/unilab/base/reset_selectors.py b/src/unilab/base/reset_selectors.py index 54ecb64bb..c33092517 100644 --- a/src/unilab/base/reset_selectors.py +++ b/src/unilab/base/reset_selectors.py @@ -59,7 +59,7 @@ def scatter(self, env_ids: torch.Tensor | ResetSelection) -> torch.Tensor: return self._mask def resolve( - self, selector: torch.Tensor | slice | ResetSelection | None, *, owner: str + self, selector: torch.Tensor | ResetSelection | None, *, owner: str ) -> torch.Tensor: """Resolve a public reset selector like ``resolve_reset_mask``. @@ -84,7 +84,7 @@ def resolve_reset_env_ids( @overload def resolve_reset_env_ids( - selector: torch.Tensor | slice | None, + selector: torch.Tensor | None, *, num_envs: int, device: torch.device, @@ -92,8 +92,18 @@ def resolve_reset_env_ids( ) -> torch.Tensor: ... +@overload +def resolve_reset_env_ids( + selector: ResetSelection | None, + *, + num_envs: int, + device: torch.device, + owner: str, +) -> torch.Tensor | ResetSelection: ... + + def resolve_reset_env_ids( - selector: torch.Tensor | slice | ResetSelection | None, + selector: torch.Tensor | ResetSelection | None, *, num_envs: int, device: torch.device, @@ -101,9 +111,10 @@ def resolve_reset_env_ids( ) -> torch.Tensor | ResetSelection: """Normalize a public reset selector to an int64 row-index tensor. - ``None`` selects every row; a slice selects ``arange(num_envs)[slice]``; an - integer tensor is validated and moved onto ``device``. Fixed-width bool - masks are rejected: the v1.4 mask form was removed in v1.5. + ``None`` selects every row; an integer tensor is validated and moved onto + ``device``. Fixed-width bool masks are rejected: the v1.4 mask form was + removed in v1.5. Slice selectors are rejected as well (issue #2619): pass + explicit row indices, e.g. ``torch.arange(num_envs)[slice]``. A ``ResetSelection`` is returned unchanged after a host-only structural check: it is the fixed-shape selector of selection-capable participants @@ -119,10 +130,14 @@ def resolve_reset_env_ids( if selector is None: return torch.arange(num_envs, dtype=torch.int64, device=device) if isinstance(selector, slice): - return torch.arange(num_envs, dtype=torch.int64, device=device)[selector] + raise TypeError( + f"{owner} env_ids must be a torch.Tensor or None; got a slice " + "(the slice selector form was removed; pass explicit row indices, " + "e.g. torch.arange(num_envs)[slice])" + ) if not isinstance(selector, torch.Tensor): raise TypeError( - f"{owner} env_ids must be a torch.Tensor, slice, or None; got {type(selector).__name__}" + f"{owner} env_ids must be a torch.Tensor or None; got {type(selector).__name__}" ) if selector.dtype == torch.bool: raise TypeError( @@ -160,7 +175,7 @@ def reset_env_ids_mask( def resolve_reset_mask( - selector: torch.Tensor | slice | ResetSelection | None, + selector: torch.Tensor | ResetSelection | None, *, num_envs: int, device: torch.device, diff --git a/src/unilab/base/torch_env.py b/src/unilab/base/torch_env.py index fc9736d02..0c74ff3e4 100644 --- a/src/unilab/base/torch_env.py +++ b/src/unilab/base/torch_env.py @@ -9,9 +9,7 @@ import abc import dataclasses -import os import sys -import time from collections.abc import Callable, Mapping from dataclasses import dataclass from os import PathLike @@ -38,6 +36,7 @@ ) from unilab.base.base import ABEnv, EnvCfg, EnvPlayCapabilities from unilab.base.cpu_runtime import apply_cpu_confinement +from unilab.base.phase_timer import PhaseTimer from unilab.base.reset_selection import ResetSelection from unilab.base.scene import SceneCfg from unilab.dtype_config import get_global_dtype @@ -67,17 +66,20 @@ def replace(self, **updates: Any) -> "TorchEnvState": return dataclasses.replace(self, **updates) -def _cpu_time() -> float: - process = os.times() - return process.user + process.system - - # Backends whose ``step_tensor`` publishes an event-backed asynchronous step # receipt (a unisim ``CompletionToken`` carried in ``info["completion"]``). # Event-backed Genesis and MJWarp CUDA paths publish a receipt per step; # synchronous host-bridge producers publish none. _EVENT_BACKED_STEP_RECEIPT_BACKENDS = frozenset({"genesis", "mjwarp"}) +# Internal fixed-shape step tuning constants (not user configuration; see +# ``TorchEnv._configure_fixed_shape_step``). The publication ring depth serves +# the off-policy double-buffer contract (a published obs_t stays stable while +# the next step writes obs_tp1); the warmup gives lazily allocated buffers +# eager executions before the CUDA graph capture. +_PUBLISHED_BUFFER_DEPTH = 2 +_AUTORESET_GRAPH_WARMUP = 2 + class TorchEnv(ABEnv): """Backend-agnostic Torch environment lifecycle. @@ -117,8 +119,9 @@ def __init__( ``cuda`` device is resolved to the current device index. Raises: - ValueError: If ``finite_validation_interval`` is not a positive - integer. + ValueError: If ``finite_validation_mode`` is not ``'strict'`` or + ``'deferred'``, or ``finite_validation_interval`` is not a + positive integer. """ apply_cpu_confinement(cfg.cpu_ids) self._cfg = cfg @@ -147,7 +150,14 @@ def __init__( self._rgb_array_renderer_ready = False self._nan_guard: "NanGuard | None" = None self._pending_backend_step_completion: CompletionToken | None = None - self._defer_finite_validation = bool(getattr(cfg, "defer_finite_validation", True)) + # ``finite_validation_mode`` is the single public validation + # strictness switch; it is declared on ``ManagerBasedRlEnvCfg`` and + # soft-read here so plain ``EnvCfg`` subclasses keep the deferred + # default. + mode = getattr(cfg, "finite_validation_mode", "deferred") + if mode not in ("strict", "deferred"): + raise ValueError(f"finite_validation_mode must be 'strict' or 'deferred', got {mode!r}") + self._defer_finite_validation = mode == "deferred" interval = getattr(cfg, "finite_validation_interval", 1) if isinstance(interval, bool) or not isinstance(interval, int) or interval <= 0: raise ValueError("finite_validation_interval must be a positive integer") @@ -159,19 +169,28 @@ def __init__( self._configure_fixed_shape_step(cfg) def _configure_fixed_shape_step(self, cfg: EnvCfg) -> None: - """Resolve the opt-in sync-free / graph-capturable step switches. - - - ``masked_autoreset``: autoreset runs every step as a fixed-shape - ``reset(ResetSelection)`` under the device done mask (no ``any()`` - gate, no ``nonzero``); requires deferred finite validation. - - ``persistent_step_buffers``: the working step carriers (observations, - reward, terminated, truncated) are allocated once and updated with - ``copy_``; consumers receive a ring of ``published_buffer_depth`` - persistent slots (``0`` publishes the working carriers themselves). - - ``autoreset_cuda_graph``: the masked autoreset segment is captured into - one CUDA graph after ``autoreset_graph_warmup`` eager executions and - replayed afterwards; requires both switches above, a CUDA device and - a backend declaring ``external_graph_capture``. + """Resolve the sync-free / graph-capturable step switch. + + ``autoreset_cuda_graph`` is the single public switch (default ``true`` + on the public ``ManagerBasedRlEnvCfg``). Enabling it + implies the whole fixed-shape chain: ``masked_autoreset`` (the + autoreset runs every step as a fixed-shape ``reset(ResetSelection)`` + under the device done mask — no ``any()`` gate, no ``nonzero``), + ``persistent_step_buffers`` (the working step carriers — observations, + reward, terminated, truncated — are allocated once, updated with + ``copy_``, and published through a ring of persistent slots), and + deferred finite validation (fail-closed checks publish at the poll + boundary). On CUDA the masked autoreset segment is captured into one + CUDA graph after ``_AUTORESET_GRAPH_WARMUP`` eager executions and + replayed afterwards; this requires a backend declaring + ``external_graph_capture``. + + The intermediate chain values are no longer public configuration. + They are still read here through a soft ``getattr`` protocol so tests + can inject a middle state (masked autoreset without graph capture or + ring depth ``0`` / warmup ``1``) by setting attributes on their own + stub configs; the Hydra materialization path fails closed on these + undeclared keys. """ def flag(name: str) -> bool: @@ -186,11 +205,19 @@ def bounded_int(name: str, default: int, minimum: int) -> int: raise ValueError(f"{name} must be an integer >= {minimum}, got {value!r}") return value - self._masked_autoreset = flag("masked_autoreset") - self._persistent_step_buffers = flag("persistent_step_buffers") self._autoreset_cuda_graph = flag("autoreset_cuda_graph") - self._published_buffer_depth = bounded_int("published_buffer_depth", 2, 0) - self._autoreset_graph_warmup = bounded_int("autoreset_graph_warmup", 2, 1) + # ``autoreset_cuda_graph`` implies the chain; the soft reads below are + # the documented test-only injection for intermediate states. + self._masked_autoreset = self._autoreset_cuda_graph or flag("masked_autoreset") + self._persistent_step_buffers = self._autoreset_cuda_graph or flag( + "persistent_step_buffers" + ) + self._published_buffer_depth = bounded_int( + "published_buffer_depth", _PUBLISHED_BUFFER_DEPTH, 0 + ) + self._autoreset_graph_warmup = bounded_int( + "autoreset_graph_warmup", _AUTORESET_GRAPH_WARMUP, 1 + ) capture_mode = getattr(cfg, "autoreset_graph_capture_mode", "thread_local") if capture_mode not in ("global", "thread_local", "relaxed"): raise ValueError( @@ -200,18 +227,11 @@ def bounded_int(name: str, default: int, minimum: int) -> int: self._autoreset_graph_capture_mode = capture_mode if self._masked_autoreset and not self._defer_finite_validation: raise ValueError( - "masked_autoreset requires defer_finite_validation: its fail-closed checks " - "are published at the deferred validation boundary" + "masked_autoreset requires finite_validation_mode='deferred': its " + "fail-closed checks are published at the deferred validation boundary" ) - if self._autoreset_cuda_graph: - if not (self._masked_autoreset and self._persistent_step_buffers): - raise ValueError( - "autoreset_cuda_graph requires masked_autoreset and persistent_step_buffers" - ) - if self._device.type != "cuda": - raise ValueError( - f"autoreset_cuda_graph requires a CUDA env device, got {self._device}" - ) + if self._autoreset_cuda_graph and self._device.type != "cuda": + raise ValueError(f"autoreset_cuda_graph requires a CUDA env device, got {self._device}") self._reset_selection: ResetSelection | None = ( ResetSelection.allocate(self._num_envs, self._device) if self._masked_autoreset @@ -439,41 +459,33 @@ def step(self, actions: torch.Tensor) -> TorchEnvState: Returns: The updated ``TorchEnvState``. """ - started = time.perf_counter() - cpu_started = _cpu_time() - phase = time.perf_counter() - self._validate_action_contract(actions) - self._record_action_finite(actions) - action_validate_ms = (time.perf_counter() - phase) * 1000.0 + timer = PhaseTimer() + with timer.phase("action_validate", cpu=False): + self._validate_action_contract(actions) + self._record_action_finite(actions) self._bind_tensor_runtime() if self._state is None: self.init_state() assert self._state is not None - phase = time.perf_counter() - phase_cpu = _cpu_time() - # Action terms publish into one device-resident control verdict. Clear - # both publication carriers before each new action is processed. - self._control_finite_valid.fill_(True) - self._pending_control_finite_check = None - ctrl = self.apply_action(actions, self._state) - guard_defer = self._defer_finite_validation - if self._nan_guard is not None: - guard_defer = self._defer_finite_validation or self._nan_guard.defer_validation - self._validate_control_with_nan_guard(ctrl, defer=guard_defer) - apply_action_ms = (time.perf_counter() - phase) * 1000.0 - apply_action_cpu_ms = (_cpu_time() - phase_cpu) * 1000.0 - - phase = time.perf_counter() - phase_cpu = _cpu_time() - self._pending_backend_step_completion = None - # A receipt describes only the transition being published. Remove the - # previous one before the backend call so unsupported or failed steps - # cannot expose stale completion metadata. - self._state.info.pop("completion", None) - backend_result = self.advance_physics(ctrl) - step_core_ms = (time.perf_counter() - phase) * 1000.0 - step_core_cpu_ms = (_cpu_time() - phase_cpu) * 1000.0 + with timer.phase("apply_action"): + # Action terms publish into one device-resident control verdict. Clear + # both publication carriers before each new action is processed. + self._control_finite_valid.fill_(True) + self._pending_control_finite_check = None + ctrl = self.apply_action(actions, self._state) + # The attached NanGuard follows the env's validation strictness + # (set_nan_guard derives its defer mode), so the env mode decides. + guard_defer = self._defer_finite_validation + self._validate_control_with_nan_guard(ctrl, defer=guard_defer) + + with timer.phase("step_core"): + self._pending_backend_step_completion = None + # A receipt describes only the transition being published. Remove the + # previous one before the backend call so unsupported or failed steps + # cannot expose stale completion metadata. + self._state.info.pop("completion", None) + backend_result = self.advance_physics(ctrl) pending_completion = self._backend_step_completion(backend_result) # Publish the receipt before update_state so public transition consumers # can only read it after the update boundary succeeds. @@ -484,9 +496,8 @@ def step(self, actions: torch.Tensor) -> TorchEnvState: self._state.truncated.fill_(False) self._clear_step_final_observation() - phase = time.perf_counter() - phase_cpu = _cpu_time() - self._state = self.update_state(self._state) + with timer.phase("update_state"): + self._state = self.update_state(self._state) finally: self._pending_backend_step_completion = None if self._nan_guard is not None: @@ -501,29 +512,14 @@ def step(self, actions: torch.Tensor) -> TorchEnvState: ) if nan_ids is not None: self._nan_guard.dump(nan_ids, self._resolve_nan_guard_model_file(), transition_step) - if not self._defer_finite_validation and self._nan_guard.defer_validation: - deferred_nan_ids = self._nan_guard.poll(step=self.step_counter, force=True) - if deferred_nan_ids is not None: - self._nan_guard.dump( - deferred_nan_ids, - self._resolve_nan_guard_model_file(), - transition_step, - ) - raise ValueError( - "NanGuard detected NaN or Inf for environments " - f"{deferred_nan_ids[:10].detach().cpu().tolist()}" - ) self._validate_state(self._state) - update_state_ms = (time.perf_counter() - phase) * 1000.0 - update_state_cpu_ms = (_cpu_time() - phase_cpu) * 1000.0 if self.produces_completion_receipts: # An event-backed asynchronous step drains its device work at the # first update-state host read; re-attribute that measured window # to the backend step phase so update_state reports manager work. drain_ms = self._collect_step_core_drain_ms(self._state) if drain_ms > 0.0: - step_core_ms += drain_ms - update_state_ms = max(update_state_ms - drain_ms, 0.0) + timer.move_wall("update_state", "step_core", drain_ms) self._state.info["steps"] += 1 self.step_counter += 1 @@ -532,8 +528,6 @@ def step(self, actions: torch.Tensor) -> TorchEnvState: truncated = self._compute_truncated(self._state) self._state.truncated.logical_or_(truncated) - phase = time.perf_counter() - phase_cpu = _cpu_time() # Preserve the semantic autoreset gate: an empty done mask must not # consume reset RNG or publish a terminal carrier. When update_state # made no host read of the step outputs, this gate is the first @@ -542,21 +536,18 @@ def step(self, actions: torch.Tensor) -> TorchEnvState: # reset bookkeeping. # The fixed-shape autoreset has no gate: it runs every step under the # device done mask, so nothing here reads the device. - did_reset = self._autoreset and ( - self._masked_autoreset or bool((self._state.terminated | self._state.truncated).any()) - ) - step_core_ms += (time.perf_counter() - phase) * 1000.0 - step_core_cpu_ms += (_cpu_time() - phase_cpu) * 1000.0 - - phase = time.perf_counter() - phase_cpu = _cpu_time() - if did_reset: - if self._masked_autoreset: - self._run_masked_autoreset() - else: - self._reset_done_envs(done_active=True) - reset_done_ms = (time.perf_counter() - phase) * 1000.0 - reset_done_cpu_ms = (_cpu_time() - phase_cpu) * 1000.0 + with timer.phase("step_core"): + did_reset = self._autoreset and ( + self._masked_autoreset + or bool((self._state.terminated | self._state.truncated).any()) + ) + + with timer.phase("reset_done"): + if did_reset: + if self._masked_autoreset: + self._run_masked_autoreset() + else: + self._reset_done_envs(done_active=True) self.poll_finite_validation() timing = self._state.info.setdefault("timing", {}) @@ -564,25 +555,10 @@ def step(self, actions: torch.Tensor) -> TorchEnvState: # The fixed-shape path has no host row count or per-phase split. self._clear_reset_done_detail_timing(timing) if did_reset: - timing["reset_done_reset_call_ms"] = reset_done_ms - timing["env_step_total_ms"] = (time.perf_counter() - started) * 1000.0 - timing["action_validate_ms"] = action_validate_ms - timing["apply_action_ms"] = apply_action_ms - timing["apply_action_cpu_ms"] = apply_action_cpu_ms - timing["step_core_ms"] = step_core_ms - timing["step_core_cpu_ms"] = step_core_cpu_ms - timing["update_state_ms"] = update_state_ms - timing["update_state_cpu_ms"] = update_state_cpu_ms - timing["reset_done_ms"] = reset_done_ms - timing["reset_done_cpu_ms"] = reset_done_cpu_ms - timing["env_step_other_cpu_ms"] = max( - (_cpu_time() - cpu_started) * 1000.0 - - apply_action_cpu_ms - - step_core_cpu_ms - - update_state_cpu_ms - - reset_done_cpu_ms, - 0.0, - ) + timing["reset_done_reset_call_ms"] = timer.wall_ms("reset_done") + timing["env_step_total_ms"] = timer.total_wall_ms() + timer.publish(timing) + timing["env_step_other_cpu_ms"] = timer.residual_cpu_ms() if isinstance(backend_result, dict): for key, value in backend_result.get("timing", {}).items(): timing[f"backend_{key}"] = value @@ -663,11 +639,7 @@ def _step_physics(self, ctrl: torch.Tensor) -> dict[str, Any] | None: for substep in range(self._cfg.sim_substeps): if substep > 0: ctrl = self.apply_substep_action() - self._validate_control_with_nan_guard( - ctrl, - defer=self._defer_finite_validation - or (self._nan_guard is not None and self._nan_guard.defer_validation), - ) + self._validate_control_with_nan_guard(ctrl, defer=self._defer_finite_validation) result = self._backend.step_tensor(ctrl, 1) completion = self._backend_step_completion(result) result_publishes_receipts = completion is not None @@ -913,7 +885,7 @@ def _reset_done_envs(self, *, done_active: bool = False) -> None: edge (backend inputs stay fixed-width masks). """ assert self._state is not None - reset_total_started = time.perf_counter() + timer = PhaseTimer(cpu=False) detail_timing = {key: 0.0 for key in RESET_DONE_DETAIL_TIMING_KEYS} done = self._state.terminated | self._state.truncated if not done_active and not bool(done.any()): @@ -922,53 +894,43 @@ def _reset_done_envs(self, *, done_active: bool = False) -> None: self._reset_done_count.copy_(done.sum()) done_env_ids = torch.nonzero(done, as_tuple=False).flatten() - terminal_started = time.perf_counter() # The nonzero above already published the done-row count to the host; # reuse it instead of reading the device counter back for timing. - detail_timing["reset_done_count"] = float(done_env_ids.numel()) - self._state.info["steps"].masked_fill_(done, 0) - final_observation = self._ensure_final_observation_scratch() - for name, values in self._state.obs.items(): - final_observation[name].copy_(values) - self._state.final_observation = final_observation - detail_timing["reset_done_terminal_obs_ms"] = ( - time.perf_counter() - terminal_started - ) * 1000.0 - - reset_started = time.perf_counter() - self._autoreset_reset_active = True - self._autoreset_scattered_reset_obs = False - try: - new_obs, reset_info = self.reset(done_env_ids) - finally: - self._autoreset_reset_active = False - detail_timing["reset_done_reset_call_ms"] = (time.perf_counter() - reset_started) * 1000.0 + with timer.phase("reset_done_terminal_obs"): + detail_timing["reset_done_count"] = float(done_env_ids.numel()) + self._state.info["steps"].masked_fill_(done, 0) + final_observation = self._ensure_final_observation_scratch() + for name, values in self._state.obs.items(): + final_observation[name].copy_(values) + self._state.final_observation = final_observation + + with timer.phase("reset_done_reset_call"): + self._autoreset_reset_active = True + self._autoreset_scattered_reset_obs = False + try: + new_obs, reset_info = self.reset(done_env_ids) + finally: + self._autoreset_reset_active = False collected = self._collect_reset_backend_timing_ms() detail_timing.update( {key: value for key, value in collected.items() if key in detail_timing} ) - scatter_started = time.perf_counter() - if not self._autoreset_scattered_reset_obs: - self._validate_reset_observation_mask(new_obs) - for name, values in new_obs.items(): - self._state.obs[name].copy_( - torch.where(done[:, None], values, self._state.obs[name]) - ) - self._scatter_reset_info_mask(reset_info, done) - detail_timing["reset_done_obs_scatter_ms"] = ( - time.perf_counter() - scatter_started - ) * 1000.0 + with timer.phase("reset_done_obs_scatter"): + if not self._autoreset_scattered_reset_obs: + self._validate_reset_observation_mask(new_obs) + for name, values in new_obs.items(): + self._state.obs[name].copy_( + torch.where(done[:, None], values, self._state.obs[name]) + ) + self._scatter_reset_info_mask(reset_info, done) detail_timing["reset_done_info_scatter_ms"] = 0.0 - measured = ( - detail_timing["reset_done_terminal_obs_ms"] - + detail_timing["reset_done_reset_call_ms"] - + detail_timing["reset_done_obs_scatter_ms"] - + detail_timing["reset_done_info_scatter_ms"] - ) + detail_timing["reset_done_terminal_obs_ms"] = timer.wall_ms("reset_done_terminal_obs") + detail_timing["reset_done_reset_call_ms"] = timer.wall_ms("reset_done_reset_call") + detail_timing["reset_done_obs_scatter_ms"] = timer.wall_ms("reset_done_obs_scatter") detail_timing["reset_done_internal_gap_ms"] = ( - time.perf_counter() - reset_total_started - ) * 1000.0 - measured + timer.total_wall_ms() - timer.attributed_wall_ms() + ) timing = self._state.info.setdefault("timing", {}) self._clear_reset_done_detail_timing(timing) @@ -1446,12 +1408,15 @@ def set_nan_guard(self, guard: NanGuard) -> None: """Attach the owner-provided diagnostic guard. Detection stays device-resident. Artifact export occurs only at the - guard's explicit abnormal-diagnostic host boundary. + guard's explicit abnormal-diagnostic host boundary. The guard's defer + mode is not owner configuration: it follows the env's single + validation strictness (``finite_validation_mode``). """ from unilab.training.tensor_diagnostics import NanGuard as _NanGuard if not isinstance(guard, _NanGuard): raise TypeError(f"TorchEnv nan guard must be NanGuard, got {type(guard).__name__}") + guard.set_defer_validation(self._defer_finite_validation) self._nan_guard = guard def _resolve_nan_guard_model_file(self) -> str: diff --git a/src/unilab/conf/appo/config.yaml b/src/unilab/conf/appo/config.yaml index 6b9ba78be..64f9f6a29 100644 --- a/src/unilab/conf/appo/config.yaml +++ b/src/unilab/conf/appo/config.yaml @@ -80,11 +80,8 @@ training: buffer_size: 100 max_envs_to_dump: 5 output_dir: null - defer_validation: true check_interval: 32 capture_interval: 32 - defer_finite_validation: true - finite_validation_interval: 32 log_root: null log_dir: null play_only: false @@ -102,6 +99,15 @@ training: cam_tracking_extra_envs: 2 replay_queue_size: null +# Finite (NaN/Inf) validation strictness, owned by the env: 'deferred' +# (default) latches verdicts on-device and publishes once per +# finite_validation_interval steps; 'strict' raises at the check site. +# Debug: env.finite_validation_interval=1 raises at every poll boundary; +# env.finite_validation_mode=strict gives immediate checks everywhere, +# including the MJWarp backend operand checks. +env: + finite_validation_interval: 32 + interactive: action_mode: zero policy_obs_mode: auto diff --git a/src/unilab/conf/flashsac/config.yaml b/src/unilab/conf/flashsac/config.yaml index 06aa3f0ec..2a17625d5 100644 --- a/src/unilab/conf/flashsac/config.yaml +++ b/src/unilab/conf/flashsac/config.yaml @@ -100,14 +100,8 @@ training: buffer_size: 100 max_envs_to_dump: 5 output_dir: null - defer_validation: true check_interval: 32 capture_interval: 32 - # Collect finite verdicts on-device and publish once per 32-step cycle. - # Debug: set defer_finite_validation=false, nan_guard.defer_validation=false, - # and nan_guard.capture_interval=1 for immediate checks/full snapshots. - defer_finite_validation: true - finite_validation_interval: 32 use_amp: true play_only: false no_play: false @@ -150,6 +144,15 @@ training: compile_threads: auto set_env_vars: true +# Finite (NaN/Inf) validation strictness, owned by the env: 'deferred' +# (default) latches verdicts on-device and publishes once per +# finite_validation_interval steps; 'strict' raises at the check site. +# Debug: env.finite_validation_interval=1 raises at every poll boundary; +# env.finite_validation_mode=strict gives immediate checks everywhere, +# including the MJWarp backend operand checks. +env: + finite_validation_interval: 32 + interactive: action_mode: zero policy_obs_mode: auto diff --git a/src/unilab/conf/ppo/config.yaml b/src/unilab/conf/ppo/config.yaml index a50932f9b..9507e0bb8 100644 --- a/src/unilab/conf/ppo/config.yaml +++ b/src/unilab/conf/ppo/config.yaml @@ -80,11 +80,8 @@ training: buffer_size: 100 max_envs_to_dump: 5 output_dir: null - defer_validation: true check_interval: 32 capture_interval: 32 - defer_finite_validation: true - finite_validation_interval: 32 play_only: false no_play: false tf32_matmul: ieee @@ -104,6 +101,15 @@ training: num_timesteps: null log_dir: null +# Finite (NaN/Inf) validation strictness, owned by the env: 'deferred' +# (default) latches verdicts on-device and publishes once per +# finite_validation_interval steps; 'strict' raises at the check site. +# Debug: env.finite_validation_interval=1 raises at every poll boundary; +# env.finite_validation_mode=strict gives immediate checks everywhere, +# including the MJWarp backend operand checks. +env: + finite_validation_interval: 32 + interactive: action_mode: zero policy_obs_mode: auto diff --git a/src/unilab/conf/sac/config.yaml b/src/unilab/conf/sac/config.yaml index 98268eeaa..ee8b55b62 100644 --- a/src/unilab/conf/sac/config.yaml +++ b/src/unilab/conf/sac/config.yaml @@ -76,11 +76,8 @@ training: buffer_size: 100 max_envs_to_dump: 5 output_dir: null - defer_validation: true check_interval: 32 capture_interval: 32 - defer_finite_validation: true - finite_validation_interval: 32 use_amp: true play_only: false no_play: false @@ -121,6 +118,15 @@ training: compile_threads: auto set_env_vars: true +# Finite (NaN/Inf) validation strictness, owned by the env: 'deferred' +# (default) latches verdicts on-device and publishes once per +# finite_validation_interval steps; 'strict' raises at the check site. +# Debug: env.finite_validation_interval=1 raises at every poll boundary; +# env.finite_validation_mode=strict gives immediate checks everywhere, +# including the MJWarp backend operand checks. +env: + finite_validation_interval: 32 + interactive: action_mode: zero policy_obs_mode: auto diff --git a/src/unilab/envs/manager_based_rl_env.py b/src/unilab/envs/manager_based_rl_env.py index 14658894d..a3cba5b88 100644 --- a/src/unilab/envs/manager_based_rl_env.py +++ b/src/unilab/envs/manager_based_rl_env.py @@ -46,6 +46,7 @@ SceneTensorReadPlan, SceneTensorReadSpec, ) +from unilab.base.phase_timer import PhaseTimer from unilab.base.reset_selection import ResetSelection from unilab.base.reset_selectors import ResetMaskBuffer, resolve_reset_env_ids from unilab.base.reset_state import ResetStateTransaction @@ -131,29 +132,44 @@ class ManagerBasedRlEnvCfg(EnvCfg): scale_rewards_by_dt: bool = True policy_observation_group: str = "policy" critic_observation_group: str | None = None - defer_finite_validation: bool = True finite_validation_interval: int = 1 - # Fixed-shape, host-sync-free autoreset (opt-in). Every reset participant - # must accept a ``ResetSelection`` (see unilab.base.reset_selection); the - # env reports unsupported participants by name at init_state. The reset - # lifecycle runs on every step (an all-false mask is a no-op), so - # full-width reset samplers also draw RNG on steps without a done row and - # same-seed trajectories differ from the selected-row lifecycle; + # Single public validation-strictness switch for all finite (NaN/Inf) + # checks. "deferred" (default) latches every fail-closed verdict on the + # device and publishes them at one host poll every + # ``finite_validation_interval`` steps; "strict" raises at each check + # site immediately (host-syncing). The concept derives the env's deferred + # finite validation, the NaN guard defer mode (an attached guard follows + # the env), and the backend options (MJWarp: strict keeps its per-call + # operand check, deferred latches its fail-closed checks into device + # verdicts). ``autoreset_cuda_graph`` requires the deferred mode: the + # fixed-shape chain publishes its fail-closed checks at the poll boundary. + finite_validation_mode: str = "deferred" + # Single public switch for the fixed-shape, host-sync-free autoreset fast + # path, on by default (issue #2621). ``true`` implies the whole chain: the + # autoreset runs on every step as a full-width ``reset(ResetSelection)`` + # under the device done mask (no ``any()`` gate, no ``nonzero``), the step + # carriers are persistent and published through a ring of fixed slots, and + # fail-closed finite checks are deferred to the poll boundary. On CUDA the + # masked autoreset segment is additionally captured into one CUDA graph and + # replayed afterwards. Every reset participant must accept a + # ``ResetSelection`` (see unilab.base.reset_selection); the env reports + # unsupported participants by name at init_state. The reset lifecycle + # runs on every step (an all-false mask is a no-op), so full-width reset + # samplers also draw RNG on steps without a done row and same-seed + # trajectories differ from the selected-row lifecycle; # ``final_observation`` is published every step and is meaningful on done - # rows only. - masked_autoreset: bool = False - # Persistent step carriers updated in place (opt-in). Consumers receive a - # ring of ``published_buffer_depth`` persistent publication slots; a - # published state stays valid for ``published_buffer_depth - 1`` further - # steps. ``0`` publishes the working carriers themselves (no copy; the - # next step overwrites them in place). - persistent_step_buffers: bool = False - published_buffer_depth: int = 2 - # Capture the masked autoreset segment into one CUDA graph (opt-in; - # requires masked_autoreset, persistent_step_buffers, a CUDA collector - # device and a backend declaring ``external_graph_capture``). - autoreset_cuda_graph: bool = False - autoreset_graph_warmup: int = 2 + # rows only. The intermediate chain switches (``masked_autoreset`` and + # ``persistent_step_buffers``) are internal, derived from this one; the + # publication ring depth and the graph warmup are owned as constants by + # ``TorchEnv``; deferred finite validation comes from + # ``finite_validation_mode='deferred'`` (required, validated below). + # + # Unsupported combinations fail closed at startup instead of silently + # falling back to eager: a non-CUDA env device, or a backend without the + # ``external_graph_capture`` capability. Set the switch explicitly to + # ``false`` to opt out — the escape hatch for debugging or for falling + # back when a graph-related problem is suspected. + autoreset_cuda_graph: bool = True autoreset_graph_capture_mode: str = "thread_local" def validate(self) -> None: @@ -220,33 +236,15 @@ def validate(self) -> None: "is_finite_horizon", "auto_reset", "scale_rewards_by_dt", - "defer_finite_validation", - "masked_autoreset", - "persistent_step_buffers", "autoreset_cuda_graph", ): if not isinstance(getattr(self, name), bool): raise TypeError(f"ManagerBasedRlEnvCfg {name} must be bool") - for name, minimum in (("published_buffer_depth", 0), ("autoreset_graph_warmup", 1)): - value = getattr(self, name) - if isinstance(value, bool) or not isinstance(value, int) or value < minimum: - raise ValueError(f"ManagerBasedRlEnvCfg {name} must be an integer >= {minimum}") if self.autoreset_graph_capture_mode not in ("global", "thread_local", "relaxed"): raise ValueError( "ManagerBasedRlEnvCfg autoreset_graph_capture_mode must be 'global', " "'thread_local' or 'relaxed'" ) - if self.masked_autoreset and not self.defer_finite_validation: - raise ValueError( - "ManagerBasedRlEnvCfg masked_autoreset requires defer_finite_validation" - ) - if self.autoreset_cuda_graph and not ( - self.masked_autoreset and self.persistent_step_buffers - ): - raise ValueError( - "ManagerBasedRlEnvCfg autoreset_cuda_graph requires masked_autoreset and " - "persistent_step_buffers" - ) if ( isinstance(self.finite_validation_interval, bool) or not isinstance(self.finite_validation_interval, int) @@ -255,6 +253,17 @@ def validate(self) -> None: raise ValueError( "ManagerBasedRlEnvCfg finite_validation_interval must be a positive integer" ) + if self.finite_validation_mode not in ("strict", "deferred"): + raise ValueError( + "ManagerBasedRlEnvCfg finite_validation_mode must be 'strict' or " + f"'deferred', got {self.finite_validation_mode!r}" + ) + if self.autoreset_cuda_graph and self.finite_validation_mode != "deferred": + raise ValueError( + "ManagerBasedRlEnvCfg autoreset_cuda_graph requires " + "finite_validation_mode='deferred': its fail-closed checks are " + "published at the deferred validation boundary" + ) if not isinstance(self.policy_observation_group, str) or not self.policy_observation_group: raise ValueError("policy_observation_group must be a non-empty string") if self.critic_observation_group is not None: @@ -1471,126 +1480,132 @@ def update_state(self, state: TorchEnvState) -> TorchEnvState: # the action-phase packet first; in-phase mutations explicitly invalidate # it below. timing = state.info.setdefault("timing", {}) - boundary_started = time.perf_counter() + timer = PhaseTimer(cpu=False) + timer.mark("update_state_read_boundary") self.scene._invalidate_state_reads() with self.scene._scoped_state_reads(): read_plan = self.scene.read_plan if read_plan is not None and not read_plan.ready: read_plan.refresh() - boundary_ms = (time.perf_counter() - boundary_started) * 1000.0 - timing["update_state_read_boundary_ms"] = boundary_ms + timing["update_state_read_boundary_ms"] = timer.span_ms("update_state_read_boundary") return self._update_state_in_read_phase(state) def _update_state_in_read_phase(self, state: TorchEnvState) -> TorchEnvState: timing = state.info.setdefault("timing", {}) - update_started_ns = time.perf_counter_ns() - prelude_started_ns = time.perf_counter_ns() - queue_started = time.perf_counter() - phase_started = time.perf_counter() - log: dict[str, Any] = {} - state.info["log"] = log - self.extras = state.info - - self.common_step_counter = self.step_counter + 1 - self._sim_step_counter = self.common_step_counter * self._cfg.sim_substeps - timing["update_state_prelude_ms"] = (time.perf_counter_ns() - prelude_started_ns) / 1.0e6 - - termination_compute_started = time.perf_counter() - self._advance_episode_steps(state.info["steps"]) - self._compute_termination_and_reset_flags() - termination_compute_ms = (time.perf_counter() - termination_compute_started) * 1000.0 - termination_timing = getattr(self.termination_manager, "last_step_timing_ms", {}) - timing.update(termination_timing) - state.info["update_state_timing"] = dict(termination_timing) - # Termination is normally the first update-state reduction after the - # backend step. Its wall time can therefore drain pre-update GPU work; - # keep that synchronization attribution separate from Manager work. - timing["update_state_queue_drain_ms"] = (time.perf_counter() - queue_started) * 1000.0 - timing["update_state_termination_ms"] = (time.perf_counter() - phase_started) * 1000.0 - timing["update_state_termination_host_ms"] = termination_compute_ms - phase_started = time.perf_counter() - timing["update_state_reset_flags_ms"] = (time.perf_counter() - phase_started) * 1000.0 - phase_started = time.perf_counter() - - self.reward_buf = self._compute_rewards() - reward_timing = getattr(self.reward_manager, "last_step_timing_ms", {}) - state.info["update_state_timing"].update(reward_timing) - timing.update(reward_timing) - timing["update_state_reward_ms"] = (time.perf_counter() - phase_started) * 1000.0 - phase_started = time.perf_counter() - self._reward_log_names = self.reward_manager.step_reward_names - self._reward_log_means = self.reward_manager.step_reward_means - - if self._cfg.sim_substeps == 1: - self.metrics_manager.compute_substep() - self._compute_metrics() - timing["update_state_metrics_ms"] = (time.perf_counter() - phase_started) * 1000.0 - phase_started = time.perf_counter() - - events_started = time.perf_counter() - self._apply_runtime_events() - timing["update_state_events_ms"] = (time.perf_counter() - events_started) * 1000.0 - - command_preflight_started = time.perf_counter() - step_reset_context = self._prepare_command_update() - timing["update_state_command_preflight_ms"] = ( - time.perf_counter() - command_preflight_started - ) * 1000.0 - command_started = time.perf_counter() - with step_reset_context: - self._compute_commands() - step_timing = getattr(self.command_manager, "last_step_timing_ms", {}) - state.info["update_state_timing"].update(step_timing) - timing.update(step_timing) - timing["update_state_command_compute_ms"] = (time.perf_counter() - command_started) * 1000.0 - command_epilogue_started = time.perf_counter() - read_refresh_ms = self._refresh_reads_after_command_commit() - post_compute_started = time.perf_counter() - self._post_compute_commands() - post_compute_ms = (time.perf_counter() - post_compute_started) * 1000.0 - timing["update_state_command_read_refresh_ms"] = read_refresh_ms - timing["update_state_command_post_compute_ms"] = post_compute_ms - timing["update_state_command_epilogue_ms"] = ( - time.perf_counter() - command_epilogue_started - ) * 1000.0 - timing["update_state_command_ms"] = (time.perf_counter() - phase_started) * 1000.0 - phase_started = time.perf_counter() - - manager_obs = self._compute_observations() - observation_timing = getattr(self.observation_manager, "last_step_timing_ms", {}) - state.info["update_state_timing"].update(observation_timing) - timing.update(observation_timing) - timing["update_state_observation_ms"] = (time.perf_counter() - phase_started) * 1000.0 - phase_started = time.perf_counter() - - mapped_obs = self._map_observations(manager_obs) - if not self.persistent_step_buffers: - self.obs_buf = mapped_obs - self._has_transition = True - timing["update_state_map_ms"] = (time.perf_counter() - phase_started) * 1000.0 - phase_started = time.perf_counter() + timer = PhaseTimer(cpu=False) + timer.mark("update_state_queue") + with timer.phase("update_state_termination"): + with timer.phase("update_state_prelude"): + log: dict[str, Any] = {} + state.info["log"] = log + self.extras = state.info + + self.common_step_counter = self.step_counter + 1 + self._sim_step_counter = self.common_step_counter * self._cfg.sim_substeps + timing["update_state_prelude_ms"] = timer.wall_ms("update_state_prelude") + + with timer.phase("update_state_termination_host"): + self._advance_episode_steps(state.info["steps"]) + self._compute_termination_and_reset_flags() + termination_timing = getattr(self.termination_manager, "last_step_timing_ms", {}) + timing.update(termination_timing) + state.info["update_state_timing"] = dict(termination_timing) + # Termination is normally the first update-state reduction after the + # backend step. Its wall time can therefore drain pre-update GPU work; + # keep that synchronization attribution separate from Manager work. + timing["update_state_queue_drain_ms"] = timer.span_ms("update_state_queue") + timing["update_state_termination_ms"] = timer.wall_ms("update_state_termination") + timing["update_state_termination_host_ms"] = timer.wall_ms("update_state_termination_host") + + with timer.phase("update_state_reset_flags"): + pass + timing["update_state_reset_flags_ms"] = timer.wall_ms("update_state_reset_flags") + + with timer.phase("update_state_reward"): + self.reward_buf = self._compute_rewards() + reward_timing = getattr(self.reward_manager, "last_step_timing_ms", {}) + state.info["update_state_timing"].update(reward_timing) + timing.update(reward_timing) + timing["update_state_reward_ms"] = timer.wall_ms("update_state_reward") + + with timer.phase("update_state_metrics"): + self._reward_log_names = self.reward_manager.step_reward_names + self._reward_log_means = self.reward_manager.step_reward_means + + if self._cfg.sim_substeps == 1: + self.metrics_manager.compute_substep() + self._compute_metrics() + timing["update_state_metrics_ms"] = timer.wall_ms("update_state_metrics") + + with timer.phase("update_state_command"): + with timer.phase("update_state_events"): + self._apply_runtime_events() + timing["update_state_events_ms"] = timer.wall_ms("update_state_events") + + with timer.phase("update_state_command_preflight"): + step_reset_context = self._prepare_command_update() + timing["update_state_command_preflight_ms"] = timer.wall_ms( + "update_state_command_preflight" + ) - publish_started = time.perf_counter() - if self.persistent_step_buffers: - self._copy_step_outputs(state, mapped_obs) - else: - public_obs, public_reward, public_terminated, public_truncated = ( - self._publish_step_outputs(mapped_obs) + with timer.phase("update_state_command_compute"): + with step_reset_context: + self._compute_commands() + step_timing = getattr(self.command_manager, "last_step_timing_ms", {}) + state.info["update_state_timing"].update(step_timing) + timing.update(step_timing) + timing["update_state_command_compute_ms"] = timer.wall_ms( + "update_state_command_compute" ) - timing["update_state_publication_ms"] = (time.perf_counter() - publish_started) * 1000.0 - replace_started = time.perf_counter() - if self.persistent_step_buffers: - replacement = state - else: - replacement = state.replace( - obs=public_obs, - reward=public_reward, - terminated=public_terminated, - truncated=public_truncated, + with timer.phase("update_state_command_epilogue"): + read_refresh_ms = self._refresh_reads_after_command_commit() + with timer.phase("update_state_command_post_compute"): + self._post_compute_commands() + post_compute_ms = timer.wall_ms("update_state_command_post_compute") + timing["update_state_command_read_refresh_ms"] = read_refresh_ms + timing["update_state_command_post_compute_ms"] = post_compute_ms + timing["update_state_command_epilogue_ms"] = timer.wall_ms( + "update_state_command_epilogue" ) - timing["update_state_state_replace_ms"] = (time.perf_counter() - replace_started) * 1000.0 - timing["update_state_publish_ms"] = (time.perf_counter() - phase_started) * 1000.0 + timing["update_state_command_ms"] = timer.wall_ms("update_state_command") + + with timer.phase("update_state_observation"): + manager_obs = self._compute_observations() + observation_timing = getattr(self.observation_manager, "last_step_timing_ms", {}) + state.info["update_state_timing"].update(observation_timing) + timing.update(observation_timing) + timing["update_state_observation_ms"] = timer.wall_ms("update_state_observation") + + with timer.phase("update_state_map"): + mapped_obs = self._map_observations(manager_obs) + if not self.persistent_step_buffers: + self.obs_buf = mapped_obs + self._has_transition = True + timing["update_state_map_ms"] = timer.wall_ms("update_state_map") + + with timer.phase("update_state_publish"): + with timer.phase("update_state_publication"): + if self.persistent_step_buffers: + self._copy_step_outputs(state, mapped_obs) + else: + public_obs, public_reward, public_terminated, public_truncated = ( + self._publish_step_outputs(mapped_obs) + ) + timing["update_state_publication_ms"] = timer.wall_ms("update_state_publication") + + with timer.phase("update_state_state_replace"): + if self.persistent_step_buffers: + replacement = state + else: + replacement = state.replace( + obs=public_obs, + reward=public_reward, + terminated=public_terminated, + truncated=public_truncated, + ) + timing["update_state_state_replace_ms"] = timer.wall_ms("update_state_state_replace") + timing["update_state_publish_ms"] = timer.wall_ms("update_state_publish") update_children = ( timing.get("update_state_termination_ms"), timing.get("update_state_reward_ms"), @@ -1601,18 +1616,18 @@ def _update_state_in_read_phase(self, state: TorchEnvState) -> TorchEnvState: timing.get("update_state_publish_ms"), ) if all(value is not None for value in update_children): - epilogue_started_ns = time.perf_counter_ns() - # Queue drain and the same boundary inside termination are excluded: - # they drain pre-update GPU work already timed by step_core_ms. - drain_ns = int(float(timing["update_state_queue_drain_ms"]) * 1.0e6) - child_sum_ms = sum(float(value) for value in update_children) - timing["update_state_child_sum_ms"] = child_sum_ms - timing["update_state_nonattributed_ms"] = ( - time.perf_counter_ns() - update_started_ns - child_sum_ms * 1.0e6 + drain_ns - ) / 1.0e6 - timing["update_state_timing_epilogue_ms"] = ( - time.perf_counter_ns() - epilogue_started_ns - ) / 1.0e6 + with timer.phase("update_state_timing_epilogue"): + # Queue drain and the same boundary inside termination are excluded: + # they drain pre-update GPU work already timed by step_core_ms. + drain_ns = int(float(timing["update_state_queue_drain_ms"]) * 1.0e6) + child_sum_ms = sum(float(value) for value in update_children) + timing["update_state_child_sum_ms"] = child_sum_ms + timing["update_state_nonattributed_ms"] = ( + timer.total_wall_ms() - child_sum_ms + drain_ns / 1.0e6 + ) + timing["update_state_timing_epilogue_ms"] = timer.wall_ms( + "update_state_timing_epilogue" + ) return replacement def _advance_episode_steps(self, steps: torch.Tensor) -> None: diff --git a/src/unilab/managers/action_manager.py b/src/unilab/managers/action_manager.py index 0695676f3..e648ee037 100644 --- a/src/unilab/managers/action_manager.py +++ b/src/unilab/managers/action_manager.py @@ -240,14 +240,14 @@ def get_term(self, name: str) -> ActionTerm: supports_reset_selection = True - def reset(self, env_ids: torch.Tensor | slice | None = None) -> dict[str, float]: + def reset(self, env_ids: torch.Tensor | None = None) -> dict[str, float]: self.clear_action_state(env_ids) return {} def reset_selection_blockers(self) -> list[str]: return selection_blockers("ActionManager", self._terms.items()) - def clear_action_state(self, env_ids: torch.Tensor | slice | None = None) -> None: + def clear_action_state(self, env_ids: torch.Tensor | None = None) -> None: """Clear manager and term action state for the selected envs without logging. Reset owners consume this boundary when they own the action reset pass. diff --git a/src/unilab/managers/command_manager.py b/src/unilab/managers/command_manager.py index 0a50e47af..1508cd6f2 100644 --- a/src/unilab/managers/command_manager.py +++ b/src/unilab/managers/command_manager.py @@ -100,13 +100,14 @@ def command(self): def reset( self, - env_ids: torch.Tensor | slice | ResetSelection | None = None, + env_ids: torch.Tensor | ResetSelection | None = None, *, publish_metrics: bool = True, ) -> dict[str, float]: if isinstance(env_ids, ResetSelection) and not self._deferred_validation_enabled(): raise RuntimeError( - f"CommandTerm '{self.name}' ResetSelection requires defer_finite_validation" + f"CommandTerm '{self.name}' ResetSelection requires deferred finite " + "validation (env.finite_validation_mode='deferred')" ) reset_mask = self._reset_mask_buffer.resolve( env_ids, @@ -137,8 +138,6 @@ def reset( getattr(self._env, "_has_transition", False) ) selected_rows = not (isinstance(env_ids, torch.Tensor) and env_ids.numel() == 0) - if isinstance(env_ids, slice): - selected_rows = bool(range(*env_ids.indices(self.num_envs))) if runtime_reset and selected_rows: self._pending_metric_count += reset_mask.sum() for (metric_name, _), selected in zip( @@ -483,13 +482,11 @@ def reset_selection_blockers(self) -> list[str]: if self._terms and not bool(getattr(self._env, "defer_finite_validation", False)): blockers.append( "CommandManager: reset metrics are read on the host unless " - "defer_finite_validation is enabled" + "finite_validation_mode='deferred'" ) return blockers - def reset( - self, env_ids: torch.Tensor | slice | ResetSelection | None = None - ) -> dict[str, float]: + def reset(self, env_ids: torch.Tensor | ResetSelection | None = None) -> dict[str, float]: """Reset command terms for the selected environment rows. Every term consumes the public ``env_ids`` row selector; the packed @@ -669,7 +666,7 @@ def __repr__(self) -> str: def get_active_iterable_terms(self, env_idx: int) -> Sequence[tuple[str, Sequence[float]]]: return [] - def reset(self, env_ids: torch.Tensor | slice | None = None) -> dict[str, float]: + def reset(self, env_ids: torch.Tensor | None = None) -> dict[str, float]: del env_ids return {} diff --git a/src/unilab/managers/curriculum_manager.py b/src/unilab/managers/curriculum_manager.py index 3056c8fbb..a94ba47d6 100644 --- a/src/unilab/managers/curriculum_manager.py +++ b/src/unilab/managers/curriculum_manager.py @@ -105,7 +105,7 @@ def reset_selection_blockers(self) -> list[str]: for name in self._term_names ] - def reset(self, env_ids: torch.Tensor | slice | None = None) -> dict[str, float]: + def reset(self, env_ids: torch.Tensor | None = None) -> dict[str, float]: """Publish curriculum state and reset term state for the selected envs.""" env_ids = resolve_reset_env_ids( env_ids, @@ -191,7 +191,7 @@ def __repr__(self) -> str: def get_active_iterable_terms(self, env_idx: int) -> Sequence[tuple[str, Sequence[float]]]: return [] - def reset(self, env_ids: torch.Tensor | slice | None = None) -> dict[str, float]: + def reset(self, env_ids: torch.Tensor | None = None) -> dict[str, float]: del env_ids return {} diff --git a/src/unilab/managers/event_manager.py b/src/unilab/managers/event_manager.py index 8f21b332b..bfaaa25a0 100644 --- a/src/unilab/managers/event_manager.py +++ b/src/unilab/managers/event_manager.py @@ -175,9 +175,7 @@ def reset_selection_blockers(self) -> list[str]: participants.extend((f"{mode}:{cfg.func!r}", cfg.func) for cfg in mode_cfg) return selection_blockers("EventManager", participants) - def reset( - self, env_ids: torch.Tensor | slice | ResetSelection | None = None - ) -> dict[str, float]: + def reset(self, env_ids: torch.Tensor | ResetSelection | None = None) -> dict[str, float]: """Clear stateful event terms for the selected environment rows.""" ids = resolve_reset_env_ids( env_ids, num_envs=self.num_envs, device=self._device, owner="EventManager.reset" diff --git a/src/unilab/managers/manager_base.py b/src/unilab/managers/manager_base.py index ff0388c72..1ca44af07 100644 --- a/src/unilab/managers/manager_base.py +++ b/src/unilab/managers/manager_base.py @@ -124,7 +124,7 @@ def active_terms(self) -> list[str] | dict[Any, list[str]]: # Methods. - def reset(self, env_ids: torch.Tensor | slice | None = None) -> dict[str, Any]: + def reset(self, env_ids: torch.Tensor | None = None) -> dict[str, Any]: """Resets manager state for the selected environment rows. The public selector follows the IsaacLab/mjlab ``env_ids`` row-index diff --git a/src/unilab/managers/metrics_manager.py b/src/unilab/managers/metrics_manager.py index 407b9a872..c93748849 100644 --- a/src/unilab/managers/metrics_manager.py +++ b/src/unilab/managers/metrics_manager.py @@ -157,7 +157,7 @@ def poll_finite_validation(self) -> bool: # Methods. - def clear_episode_state(self, env_ids: torch.Tensor | slice | None = None) -> None: + def clear_episode_state(self, env_ids: torch.Tensor | None = None) -> None: """Clear episode accumulators without publishing summaries. Reset owners consume this boundary when they own metric reset. Generic @@ -180,19 +180,20 @@ def reset_selection_blockers(self) -> list[str]: if self._term_names and not self._defer_finite_validation: blockers.append( "MetricsManager: episode summaries are read on the host unless " - "defer_finite_validation is enabled" + "finite_validation_mode='deferred'" ) return blockers - def reset( - self, env_ids: torch.Tensor | slice | ResetSelection | None = None - ) -> dict[str, float]: + def reset(self, env_ids: torch.Tensor | ResetSelection | None = None) -> dict[str, float]: """Publish episode summaries and clear state for the selected rows.""" ids = resolve_reset_env_ids( env_ids, num_envs=self.num_envs, device=self._device, owner="MetricsManager.reset" ) if isinstance(ids, ResetSelection) and not self._defer_finite_validation: - raise RuntimeError("MetricsManager ResetSelection requires defer_finite_validation") + raise RuntimeError( + "MetricsManager ResetSelection requires deferred finite " + "validation (env.finite_validation_mode='deferred')" + ) if self._defer_finite_validation: if isinstance(ids, ResetSelection): reset_mask = ids.mask @@ -396,14 +397,14 @@ def __repr__(self) -> str: def get_active_iterable_terms(self, env_idx: int) -> Sequence[tuple[str, Sequence[float]]]: return [] - def reset(self, env_ids: torch.Tensor | slice | None = None) -> dict[str, float]: + def reset(self, env_ids: torch.Tensor | None = None) -> dict[str, float]: del env_ids return {} def publish_pending_reset_metrics(self) -> dict[str, float]: return {} - def clear_episode_state(self, env_ids: torch.Tensor | slice | None = None) -> None: + def clear_episode_state(self, env_ids: torch.Tensor | None = None) -> None: del env_ids def compute_substep(self) -> None: diff --git a/src/unilab/managers/observation_manager.py b/src/unilab/managers/observation_manager.py index 08047a7e8..1a8d8dff3 100644 --- a/src/unilab/managers/observation_manager.py +++ b/src/unilab/managers/observation_manager.py @@ -427,16 +427,14 @@ def reset_selection_blockers(self) -> list[str]: ): blockers.append( "ObservationManager: NaN policies read the host unless " - "defer_finite_validation is enabled" + "finite_validation_mode='deferred'" ) return blockers def _group_cfgs_for_selection(self) -> list[Any]: return [group for group in self.cfg.values() if group is not None] - def reset( - self, env_ids: torch.Tensor | slice | ResetSelection | None = None - ) -> dict[str, float]: + def reset(self, env_ids: torch.Tensor | ResetSelection | None = None) -> dict[str, float]: """Reset observation state for the selected environment rows. Class-term state, delay/history buffers, and noise models are cleared diff --git a/src/unilab/managers/reward_manager.py b/src/unilab/managers/reward_manager.py index 9223330a3..38bf0d503 100644 --- a/src/unilab/managers/reward_manager.py +++ b/src/unilab/managers/reward_manager.py @@ -167,7 +167,7 @@ def reset_selection_blockers(self) -> list[str]: ), ) - def reset(self, env_ids: torch.Tensor | slice | None = None) -> dict[str, float]: + def reset(self, env_ids: torch.Tensor | None = None) -> dict[str, float]: """Clear stateful reward terms for the selected environment rows.""" ids = resolve_reset_env_ids( env_ids, num_envs=self.num_envs, device=self._device, owner="RewardManager.reset" diff --git a/src/unilab/managers/termination_manager.py b/src/unilab/managers/termination_manager.py index dfa9f89de..91c424f8d 100644 --- a/src/unilab/managers/termination_manager.py +++ b/src/unilab/managers/termination_manager.py @@ -115,11 +115,11 @@ def reset_selection_blockers(self) -> list[str]: if not self._deferred_validation_enabled(): blockers.append( "TerminationManager: episode counts are read on the host unless " - "defer_finite_validation is enabled" + "finite_validation_mode='deferred'" ) return blockers - def reset(self, env_ids: torch.Tensor | slice | ResetSelection | None = None) -> dict[str, int]: + def reset(self, env_ids: torch.Tensor | ResetSelection | None = None) -> dict[str, int]: """Publish episodic counts and clear terms for the selected rows.""" ids = resolve_reset_env_ids( env_ids, num_envs=self.num_envs, device=self._device, owner="TerminationManager.reset" @@ -129,7 +129,8 @@ def reset(self, env_ids: torch.Tensor | slice | ResetSelection | None = None) -> if isinstance(ids, ResetSelection): if not self._deferred_validation_enabled(): raise RuntimeError( - "TerminationManager ResetSelection requires defer_finite_validation" + "TerminationManager ResetSelection requires deferred finite " + "validation (env.finite_validation_mode='deferred')" ) counts = [(self._term_dones[key] & ids.mask).sum() for key in keys] selected_rows = True diff --git a/src/unilab/scripts/train_appo.py b/src/unilab/scripts/train_appo.py index a4107c1fb..23e2e373c 100644 --- a/src/unilab/scripts/train_appo.py +++ b/src/unilab/scripts/train_appo.py @@ -41,7 +41,7 @@ ) from unilab.training.experiment import ExperimentTracker from unilab.training.onnx_export import export_policy_onnx, verify_policy_onnx -from unilab.training.run import finite_validation_env_overrides +from unilab.training.run import reject_removed_validation_keys from unilab.training.seed import apply_configured_training_seed from unilab.training.tensor_runtime import ( resolve_recorded_collector_tensor_device, @@ -124,7 +124,7 @@ def build_appo_runner_kwargs( collector_tensor_device=select_collector_tensor_device(cfg), learner_device=_env_carrier_device(cfg, collector_device), ) - routed_env_cfg_override.update(finite_validation_env_overrides(cfg)) + reject_removed_validation_keys(cfg) runner_kwargs = { "env_name": cfg.training.task_name, @@ -383,7 +383,7 @@ def main(cfg: DictConfig) -> None: collector_tensor_device=select_collector_tensor_device(cfg), learner_device=_env_carrier_device(cfg, collector_device, runner_device=learner_device), ) - env_cfg_override.update(finite_validation_env_overrides(cfg)) + reject_removed_validation_keys(cfg) seed_info = apply_configured_training_seed(cfg, torch_runtime=True, cuda=True) tf32_matmul_policy = apply_tf32_matmul_from_cfg(cfg, device=learner_device) resolved_collector_tensor_device = resolve_recorded_collector_tensor_device( diff --git a/src/unilab/scripts/train_offpolicy.py b/src/unilab/scripts/train_offpolicy.py index f4ca25dbb..83cd9e388 100644 --- a/src/unilab/scripts/train_offpolicy.py +++ b/src/unilab/scripts/train_offpolicy.py @@ -59,7 +59,7 @@ from unilab.training.dp_config import apply_dp_rank_config from unilab.training.experiment import ExperimentTracker from unilab.training.onnx_export import export_policy_onnx, verify_policy_onnx -from unilab.training.run import finite_validation_env_overrides +from unilab.training.run import reject_removed_validation_keys from unilab.training.seed import apply_configured_training_seed from unilab.training.tensor_runtime import ( resolve_collector_tensor_placement, @@ -227,7 +227,7 @@ def _genesis_probe_para_level(sim_backend: str): def build_offpolicy_env_cfg_override(algo_name: str, cfg: DictConfig) -> dict[str, Any] | None: base = _build_offpolicy_env_cfg_override(algo_name, cfg, root_dir=Path.cwd()) base = dict(base or {}) - base.update(finite_validation_env_overrides(cfg)) + reject_removed_validation_keys(cfg) rank = current_dp_rank() from unilab.utils.device import get_default_device diff --git a/src/unilab/scripts/train_rsl_rl.py b/src/unilab/scripts/train_rsl_rl.py index 5ae000425..4a1f16b04 100644 --- a/src/unilab/scripts/train_rsl_rl.py +++ b/src/unilab/scripts/train_rsl_rl.py @@ -56,7 +56,7 @@ patch_rsl_rl_tensorboard_logging, patch_rsl_rl_wandb_writer, ) -from unilab.training.run import finite_validation_env_overrides +from unilab.training.run import reject_removed_validation_keys from unilab.training.seed import apply_configured_training_seed from unilab.training.tensor_runtime import ( resolve_recorded_collector_tensor_device, @@ -113,7 +113,7 @@ def build_ppo_env_cfg_override(cfg: DictConfig) -> dict[str, Any]: collector_tensor_device=select_collector_tensor_device(cfg), learner_device=learner_device, ) - result.update(finite_validation_env_overrides(cfg)) + reject_removed_validation_keys(cfg) if world_size > 1: explicit = OmegaConf.select(cfg, "training.dp_collector_cpu_ids", default=None) explicit = OmegaConf.to_container(explicit, resolve=True) if explicit is not None else None diff --git a/src/unilab/tasks/motion_tracking/common/manager_terms.py b/src/unilab/tasks/motion_tracking/common/manager_terms.py index 02f5c8202..ef651ab31 100644 --- a/src/unilab/tasks/motion_tracking/common/manager_terms.py +++ b/src/unilab/tasks/motion_tracking/common/manager_terms.py @@ -889,7 +889,7 @@ def anchor_ang_vel_w(self) -> torch.Tensor: def reset( self, - env_ids: torch.Tensor | slice | ResetSelection | None = None, + env_ids: torch.Tensor | ResetSelection | None = None, *, publish_metrics: bool = True, ) -> dict[str, float]: @@ -922,17 +922,10 @@ def reset( ) self.time_left.copy_(torch.where(reset_mask, sampled, self.time_left)) self._resample_command(reset_mask, fixed_shape=fixed_shape) - if ( - env_ids is None - or ( - isinstance(env_ids, slice) - and len(range(*env_ids.indices(self.num_envs))) == self.num_envs - ) - or ( - isinstance(env_ids, torch.Tensor) - and env_ids.numel() == self.num_envs - and bool(reset_mask.all()) - ) + if env_ids is None or ( + isinstance(env_ids, torch.Tensor) + and env_ids.numel() == self.num_envs + and bool(reset_mask.all()) ): # Float32 sampling may round slightly below the configured lower # bound. Keep a conservative margin before declaring any timer diff --git a/src/unilab/training/run.py b/src/unilab/training/run.py index a7b4f969f..64b1e2a27 100644 --- a/src/unilab/training/run.py +++ b/src/unilab/training/run.py @@ -85,6 +85,16 @@ def resolve_nan_guard_cfg(training_cfg: Any) -> NanGuardCfg | None: return None from unilab.training.tensor_diagnostics import NanGuardCfg + defer_validation = ( + OmegaConf.select(nan_guard_cfg, "defer_validation", default=None) + if OmegaConf.is_config(nan_guard_cfg) + else getattr(nan_guard_cfg, "defer_validation", None) + ) + if defer_validation is not None: + raise ValueError( + "training.nan_guard.defer_validation was removed from the public config " + "surface: the guard's defer mode is derived from env.finite_validation_mode" + ) return NanGuardCfg( enabled=True, buffer_size=int(getattr(nan_guard_cfg, "buffer_size", 100)), @@ -92,22 +102,27 @@ def resolve_nan_guard_cfg(training_cfg: Any) -> NanGuardCfg | None: output_dir=getattr(nan_guard_cfg, "output_dir", None), capture_interval=getattr(nan_guard_cfg, "capture_interval", 1), check_interval=getattr(nan_guard_cfg, "check_interval", 1), - defer_validation=getattr(nan_guard_cfg, "defer_validation", False), ) -def finite_validation_env_overrides(cfg: DictConfig) -> dict[str, Any]: - """Route training's finite-check policy to the environment owner.""" - deferred = OmegaConf.select(cfg, "training.defer_finite_validation", default=True) - interval = OmegaConf.select(cfg, "training.finite_validation_interval", default=1) - if not isinstance(deferred, bool): - raise TypeError("training.defer_finite_validation must be bool") - if isinstance(interval, bool) or not isinstance(interval, int) or interval <= 0: - raise ValueError("training.finite_validation_interval must be a positive integer") - return { - "defer_finite_validation": deferred, - "finite_validation_interval": interval, - } +def reject_removed_validation_keys(cfg: DictConfig) -> None: + """Fail closed on validation-strictness keys removed from the public config surface. + + Finite validation is owned by the environment: ``env.finite_validation_mode`` + (``"strict"`` or ``"deferred"``) together with ``env.finite_validation_interval`` + is the single public concept. The removed training-level pass-throughs raise + a clear config error here instead of being silently ignored. + """ + if OmegaConf.select(cfg, "training.finite_validation_interval", default=None) is not None: + raise ValueError( + "training.finite_validation_interval was removed: finite validation is " + "owned by the env; use env.finite_validation_interval instead" + ) + if OmegaConf.select(cfg, "training.nan_guard.defer_validation", default=None) is not None: + raise ValueError( + "training.nan_guard.defer_validation was removed: the guard's defer mode " + "is derived from env.finite_validation_mode" + ) def apply_env_nan_guard(env: Any, training_cfg: Any) -> None: diff --git a/src/unilab/training/tensor_diagnostics.py b/src/unilab/training/tensor_diagnostics.py index 9e19c7579..450ea5ac5 100644 --- a/src/unilab/training/tensor_diagnostics.py +++ b/src/unilab/training/tensor_diagnostics.py @@ -23,7 +23,12 @@ @dataclass(frozen=True) class NanGuardCfg: - """Pickleable configuration for the tensor finite-state guard.""" + """Pickleable configuration for the tensor finite-state guard. + + The defer mode is not part of this configuration: it is derived from the + environment's single validation strictness (``finite_validation_mode``) + when the guard is attached (``TorchEnv.set_nan_guard``). + """ enabled: bool = False buffer_size: int = 100 @@ -34,7 +39,6 @@ class NanGuardCfg: capture_interval: int = 1 # Zero means the owner must invoke poll explicitly at its cycle boundary. check_interval: int = 1 - defer_validation: bool = False def _bad_row_mask(value: torch.Tensor, num_envs: int) -> torch.Tensor: @@ -80,6 +84,9 @@ def __init__( self._buffer_idx = 0 self._buffer_full = False self._dumped = False + # Derived, not configured: TorchEnv.set_nan_guard follows the env's + # single validation strictness (finite_validation_mode). + self._defer_validation = False # Both latches live on the first input's device. They retain all bad # rows until a single explicit host poll closes the validation interval. self._device: torch.device | None = None @@ -120,7 +127,16 @@ def device_verdict(self) -> torch.Tensor | None: @property def defer_validation(self) -> bool: - return self._cfg.defer_validation + """Whether bad rows latch on-device until an explicit poll. + + Derived from the owning env's ``finite_validation_mode`` at attach + time; not an owner-facing configuration key. + """ + return self._defer_validation + + def set_defer_validation(self, enabled: bool) -> None: + """Set the defer mode from the env's single validation strictness.""" + self._defer_validation = bool(enabled) def capture_due(self, step: int) -> bool: """Allow the owner to skip even the backend state read on ordinary steps.""" @@ -200,7 +216,7 @@ def check( for mask in masks[1:]: bad_rows.logical_or_(mask) self._record_bad_rows(bad_rows) - should_defer = self._cfg.defer_validation if defer is None else bool(defer) + should_defer = self._defer_validation if defer is None else bool(defer) return None if should_defer else self.poll(step=step, force=defer is False) def check_ctrl( @@ -211,7 +227,7 @@ def check_ctrl( return None self._validate_finite_float(ctrl, "ctrl") self._record_bad_rows(_bad_row_mask(ctrl, self._num_envs)) - should_defer = self._cfg.defer_validation if defer is None else bool(defer) + should_defer = self._defer_validation if defer is None else bool(defer) return None if should_defer else self.poll(step=step, force=defer is False) def poll(self, step: int = 0, *, force: bool = False) -> torch.Tensor | None: @@ -220,7 +236,7 @@ def poll(self, step: int = 0, *, force: bool = False) -> torch.Tensor | None: return None if ( not force - and self._cfg.defer_validation + and self._defer_validation and (self._cfg.check_interval == 0 or step % self._cfg.check_interval != 0) ): return None diff --git a/tests/base/backend/test_mjwarp_backend_options.py b/tests/base/backend/test_mjwarp_backend_options.py index 0e08d1fef..8f5bf5c49 100644 --- a/tests/base/backend/test_mjwarp_backend_options.py +++ b/tests/base/backend/test_mjwarp_backend_options.py @@ -1,10 +1,17 @@ -"""MJWarp hot-path finite-check routing (#2587). +"""MJWarp hot-path finite-check routing (#2587, #2618). TorchEnv validates the final control for NaN/Inf on every step, eagerly or through deferred finite validation, so MJWarp's own blocking per-call read is a duplicate. UniLab therefore always passes ``mjwarp_validate_tensor_finite`` -explicitly (default ``False``) instead of inheriting UniSim's blocking default; -UniSim's factory maps it to the MJWarp adapter's ``validate_tensor_finite``. +explicitly instead of inheriting UniSim's blocking default; UniSim's factory +maps it to the MJWarp adapter's ``validate_tensor_finite``. + +Both MJWarp options are derived from the env's single validation strictness +(``finite_validation_mode``, issue #2618): strict mode keeps the backend's +per-call operand check immediate, deferred mode disables it and latches the +backend's fail-closed checks into device verdicts +(``mjwarp_defer_tensor_validation``) that the env publishes at its poll +boundary. The removed ``mjwarp_*`` config keys fail closed with a clear error. """ from __future__ import annotations @@ -15,6 +22,7 @@ from unilab.base.backend_factory import create_backend, env_backend_kwargs from unilab.base.base import EnvCfg +from unilab.base.registry import apply_cfg_overrides from unilab.base.scene import SceneCfg from unilab.envs.manager_based_rl_env import ManagerBasedRlEnvCfg @@ -27,24 +35,48 @@ "cfg", [ EnvCfg(), - ManagerBasedRlEnvCfg(defer_finite_validation=True), - ManagerBasedRlEnvCfg(defer_finite_validation=False), + ManagerBasedRlEnvCfg(), + ManagerBasedRlEnvCfg(autoreset_cuda_graph=True), ], - ids=["env_cfg", "deferred", "eager"], + ids=["env_cfg", "default", "autoreset_graph"], ) -def test_mjwarp_finite_check_is_explicitly_disabled_by_default(cfg: EnvCfg) -> None: - assert env_backend_kwargs(cfg)["mjwarp_validate_tensor_finite"] is False +def test_mjwarp_finite_check_is_derived_from_validation_mode(cfg: EnvCfg) -> None: + kwargs = env_backend_kwargs(cfg) + assert kwargs["mjwarp_validate_tensor_finite"] is False + assert kwargs["mjwarp_defer_tensor_validation"] is True -def test_mjwarp_finite_check_can_be_reenabled_explicitly() -> None: - cfg = ManagerBasedRlEnvCfg(mjwarp_validate_tensor_finite=True) - assert env_backend_kwargs(cfg)["mjwarp_validate_tensor_finite"] is True +def test_mjwarp_strict_mode_keeps_the_immediate_backend_check() -> None: + cfg = ManagerBasedRlEnvCfg(finite_validation_mode="strict") + kwargs = env_backend_kwargs(cfg) + assert kwargs["mjwarp_validate_tensor_finite"] is True + assert "mjwarp_defer_tensor_validation" not in kwargs -@pytest.mark.parametrize("value", [None, 1, "false", 0.0]) -def test_mjwarp_validate_tensor_finite_rejects_non_bool(value: object) -> None: - with pytest.raises(ValueError, match="mjwarp_validate_tensor_finite must be bool"): - EnvCfg(mjwarp_validate_tensor_finite=value).validate() # type: ignore[arg-type] +def test_mjwarp_derived_options_reject_unknown_mode() -> None: + cfg = ManagerBasedRlEnvCfg() + cfg.finite_validation_mode = "sometimes" + with pytest.raises(ValueError, match="finite_validation_mode must be 'strict' or 'deferred'"): + env_backend_kwargs(cfg) + + +@pytest.mark.parametrize( + "removed", ["mjwarp_validate_tensor_finite", "mjwarp_defer_tensor_validation"] +) +def test_mjwarp_removed_keys_fail_closed_as_env_overrides(removed: str) -> None: + cfg = ManagerBasedRlEnvCfg() + with pytest.raises(ValueError, match=f"has no attribute '{removed}'"): + apply_cfg_overrides(cfg, {removed: True}) + + +@pytest.mark.parametrize( + "removed", ["mjwarp_validate_tensor_finite", "mjwarp_defer_tensor_validation"] +) +def test_mjwarp_removed_keys_fail_closed_as_injected_attributes(removed: str) -> None: + cfg = ManagerBasedRlEnvCfg() + setattr(cfg, removed, True) + with pytest.raises(ValueError, match=f"{removed} was removed from the public config surface"): + env_backend_kwargs(cfg) def test_non_mjwarp_backend_accepts_the_routed_option() -> None: @@ -60,10 +92,8 @@ def test_non_mjwarp_backend_accepts_the_routed_option() -> None: backend.close() -@pytest.mark.parametrize(("validate", "raises"), [(None, False), (True, True)]) -def test_mjwarp_step_tensor_finite_check_follows_env_cfg( - validate: bool | None, raises: bool -) -> None: +@pytest.mark.parametrize(("mode", "raises"), [("deferred", False), ("strict", True)]) +def test_mjwarp_step_tensor_finite_check_follows_validation_mode(mode: str, raises: bool) -> None: pytest.importorskip("mujoco_warp") torch = pytest.importorskip("torch") warp = pytest.importorskip("warp") @@ -71,11 +101,9 @@ def test_mjwarp_step_tensor_finite_check_follows_env_cfg( if not bool(warp.get_device().is_cuda): pytest.skip("mjwarp runtime tests require an active CUDA Warp device") - # ``None`` exercises the UniLab default; both runs use eager env-level - # validation to show the backend default does not depend on deferral. - cfg = ManagerBasedRlEnvCfg(defer_finite_validation=False) - if validate is not None: - cfg.mjwarp_validate_tensor_finite = validate + # The backend check is derived from the env-level validation mode: + # deferred disables the duplicate per-call read, strict keeps it. + cfg = ManagerBasedRlEnvCfg(finite_validation_mode=mode) backend = create_backend( "mjwarp", SceneCfg(model_file=_MODEL_FILE), diff --git a/tests/base/test_phase_timer.py b/tests/base/test_phase_timer.py new file mode 100644 index 000000000..cfb64667c --- /dev/null +++ b/tests/base/test_phase_timer.py @@ -0,0 +1,85 @@ +"""Unit tests for the structured phase timer used by the env step lifecycle.""" + +from __future__ import annotations + +import pytest + +from unilab.base.phase_timer import PhaseTimer + + +def test_phase_accumulates_repeated_entries_into_one_sample() -> None: + timer = PhaseTimer(cpu=False) + with timer.phase("step_core"): + pass + first = timer.wall_ms("step_core") + with timer.phase("step_core"): + pass + + assert timer.wall_ms("step_core") >= first + + +def test_publish_writes_paired_keys_and_skips_unmeasured_cpu() -> None: + timer = PhaseTimer() + with timer.phase("action_validate", cpu=False): + pass + with timer.phase("apply_action"): + pass + + timing: dict[str, float] = {} + timer.publish(timing) + + assert set(timing) == { + "action_validate_ms", + "apply_action_ms", + "apply_action_cpu_ms", + } + assert all(value >= 0.0 for value in timing.values()) + + +def test_move_wall_reattributes_and_floors_source_at_zero() -> None: + timer = PhaseTimer(cpu=False) + with timer.phase("update_state"): + pass + with timer.phase("step_core"): + pass + update_before = timer.wall_ms("update_state") + step_before = timer.wall_ms("step_core") + + timer.move_wall("update_state", "step_core", 0.5) + assert timer.wall_ms("step_core") == pytest.approx(step_before + 0.5) + assert timer.wall_ms("update_state") == pytest.approx(max(update_before - 0.5, 0.0)) + + timer.move_wall("update_state", "step_core", 10.0) + assert timer.wall_ms("update_state") == 0.0 + + +def test_residual_cpu_is_total_minus_attributed_and_floored() -> None: + timer = PhaseTimer() + with timer.phase("apply_action"): + pass + with timer.phase("action_validate", cpu=False): + pass + + assert timer.residual_cpu_ms() >= 0.0 + assert timer.total_cpu_ms() >= 0.0 + + +def test_wall_only_timer_rejects_cpu_phases_and_cpu_totals() -> None: + timer = PhaseTimer(cpu=False) + with timer.phase("terminal_obs"): + pass + with pytest.raises(ValueError, match="wall-only timer"): + timer.total_cpu_ms() + with pytest.raises(ValueError, match="wall-only timer"): + with timer.phase("other", cpu=True): + pass + + +def test_span_ms_measures_between_mark_and_now() -> None: + timer = PhaseTimer(cpu=False) + timer.mark("queue") + with timer.phase("work"): + pass + span = timer.span_ms("queue") + + assert span >= timer.wall_ms("work") diff --git a/tests/base/test_reset_selectors.py b/tests/base/test_reset_selectors.py index 4d7b790f7..bee8155ce 100644 --- a/tests/base/test_reset_selectors.py +++ b/tests/base/test_reset_selectors.py @@ -6,6 +6,7 @@ from unilab.base.reset_selectors import ( ResetMaskBuffer, reset_env_ids_mask, + resolve_reset_env_ids, resolve_reset_mask, ) @@ -73,7 +74,7 @@ def test_resolve_ids_matches_allocating_form() -> None: buffer = ResetMaskBuffer(num_envs, torch.device("cpu")) for selector in ( None, - slice(2, 10, 3), + torch.arange(num_envs, dtype=torch.int64)[2:10:3], torch.tensor([4, 9], dtype=torch.int64), ): expected = resolve_reset_mask( @@ -83,6 +84,18 @@ def test_resolve_ids_matches_allocating_form() -> None: assert torch.equal(reused, expected) +def test_resolve_rejects_slice() -> None: + num_envs = 16 + selector = slice(2, 10, 3) + with pytest.raises(TypeError, match="slice"): + resolve_reset_env_ids(selector, num_envs=num_envs, device=torch.device("cpu"), owner="test") + buffer = ResetMaskBuffer(num_envs, torch.device("cpu")) + with pytest.raises(TypeError, match="slice"): + buffer.resolve(selector, owner="test") + with pytest.raises(TypeError, match="slice"): + resolve_reset_mask(selector, num_envs=num_envs, device=torch.device("cpu"), owner="test") + + def test_resolve_rejects_bool_mask() -> None: num_envs = 8 buffer = ResetMaskBuffer(num_envs, torch.device("cpu")) diff --git a/tests/base/test_reward_override.py b/tests/base/test_reward_override.py index 3a2669185..f0e04c219 100644 --- a/tests/base/test_reward_override.py +++ b/tests/base/test_reward_override.py @@ -34,6 +34,8 @@ def test_g1_reset_randomization_runs_for_reward_override_fixture(): "func": "unilab.tasks.locomotion.common.manager_terms.alive", "weight": 20.0, } + # Reward-override lifecycle test on the eager selected-row path on CPU. + env_cfg_override["autoreset_cuda_graph"] = False env = registry.make( "G1WalkFlat", diff --git a/tests/base/test_terrain.py b/tests/base/test_terrain.py index b1109fd06..c274af5b0 100644 --- a/tests/base/test_terrain.py +++ b/tests/base/test_terrain.py @@ -288,6 +288,8 @@ def observation(env: ManagerBasedRlEnv) -> torch.Tensor: ctrl_dt=0.02, max_episode_seconds=1.0, seed=19, + # Terrain reset test on the eager selected-row path on CPU. + autoreset_cuda_graph=False, observations={ "actor": ObservationGroupCfg(terms={"state": ObservationTermCfg(func=observation)}) }, diff --git a/tests/base/test_torch_env.py b/tests/base/test_torch_env.py index b6893eb1a..307edb5fa 100644 --- a/tests/base/test_torch_env.py +++ b/tests/base/test_torch_env.py @@ -23,7 +23,7 @@ TensorRuntimeDiagnostic, ) -import unilab.base.torch_env as torch_env_module +import unilab.base.phase_timer as phase_timer_module from unilab.base.base import EnvCfg, EnvPlayCapabilities from unilab.base.scene import SceneCfg from unilab.base.torch_env import TorchEnv, TorchEnvState @@ -37,13 +37,13 @@ class _StubCfg(EnvCfg): sim_dt: float = 0.1 # These tests exercise the eager fail-closed paths; pin the pre-flip # behavior explicitly instead of relying on the (now deferred) default. - defer_finite_validation: bool = False + finite_validation_mode: str = "strict" @dataclass class _DeferredStubCfg(_StubCfg): max_episode_seconds: float | None = 1000.0 - defer_finite_validation: bool = True + finite_validation_mode: str = "deferred" finite_validation_interval: int = 32 @@ -459,11 +459,11 @@ def test_step_done_gate_host_read_is_attributed_to_step_core(monkeypatch) -> Non wall = itertools.count(start=0.0, step=0.001) cpu = itertools.count(start=0.0, step=0.001) monkeypatch.setattr( - torch_env_module, + phase_timer_module, "time", types.SimpleNamespace(perf_counter=lambda: next(wall)), ) - monkeypatch.setattr(torch_env_module, "_cpu_time", lambda: next(cpu)) + monkeypatch.setattr(phase_timer_module, "_cpu_time", lambda: next(cpu)) env = _StubTorchEnv(terminate=False) env.init_state() @@ -484,11 +484,11 @@ def test_step_reattributes_update_state_drain_to_step_core(monkeypatch) -> None: wall = itertools.count(start=0.0, step=0.001) cpu = itertools.count(start=0.0, step=0.001) monkeypatch.setattr( - torch_env_module, + phase_timer_module, "time", types.SimpleNamespace(perf_counter=lambda: next(wall)), ) - monkeypatch.setattr(torch_env_module, "_cpu_time", lambda: next(cpu)) + monkeypatch.setattr(phase_timer_module, "_cpu_time", lambda: next(cpu)) class _DrainEnv(_StubTorchEnv): def _collect_step_core_drain_ms(self, state: TorchEnvState) -> float: @@ -760,7 +760,6 @@ def apply_substep_action(self) -> torch.Tensor: NanGuard( NanGuardCfg( enabled=True, - defer_validation=True, check_interval=32, capture_interval=0, output_dir=str(tmp_path), @@ -807,7 +806,6 @@ def test_deferred_finite_validation_polls_once_at_interval( NanGuard( NanGuardCfg( enabled=True, - defer_validation=False, check_interval=1, capture_interval=0, ), @@ -874,7 +872,6 @@ def update_state(self, state: TorchEnvState) -> TorchEnvState: NanGuard( NanGuardCfg( enabled=True, - defer_validation=True, check_interval=32, capture_interval=0, output_dir=str(tmp_path), @@ -918,7 +915,6 @@ def test_nan_guard_reads_physics_only_at_capture_boundaries( guard = NanGuard( NanGuardCfg( enabled=True, - defer_validation=True, check_interval=32, capture_interval=32, ), @@ -949,37 +945,28 @@ def counted_read() -> torch.Tensor | None: torch.testing.assert_close(guard._buffer[1], torch.full((env.num_envs, 1), 64.0)) -def test_strict_env_with_deferred_guard_polls_once_after_state( - monkeypatch: pytest.MonkeyPatch, -) -> None: - env = _StubTorchEnv(cfg=_StubCfg(max_episode_seconds=1000.0), terminate=False) - env.init_state() - env.set_nan_guard( - NanGuard( - NanGuardCfg( - enabled=True, - defer_validation=True, - check_interval=32, - capture_interval=0, - ), - num_envs=env.num_envs, - supports_state_playback=False, - ) +def test_nan_guard_defer_mode_follows_env_validation_mode() -> None: + """The guard's defer mode is derived from the env, not guard configuration.""" + deferred_env = _StubTorchEnv(cfg=_DeferredStubCfg(), terminate=False) + deferred_guard = NanGuard( + NanGuardCfg(enabled=True, check_interval=32, capture_interval=0), + num_envs=deferred_env.num_envs, + supports_state_playback=False, ) - item_calls = 0 - original_item = torch.Tensor.item + deferred_env.set_nan_guard(deferred_guard) + assert deferred_guard.defer_validation - def counted_item(tensor: torch.Tensor, *args: Any, **kwargs: Any) -> Any: - nonlocal item_calls - item_calls += 1 - return original_item(tensor, *args, **kwargs) - - monkeypatch.setattr(torch.Tensor, "item", counted_item) - env.step(_actions(env)) - assert item_calls == 1 + strict_env = _StubTorchEnv(cfg=_StubCfg(max_episode_seconds=1000.0), terminate=False) + strict_guard = NanGuard( + NanGuardCfg(enabled=True, check_interval=32, capture_interval=0), + num_envs=strict_env.num_envs, + supports_state_playback=False, + ) + strict_env.set_nan_guard(strict_guard) + assert not strict_guard.defer_validation -def test_strict_env_with_deferred_guard_dumps_bad_control(tmp_path) -> None: +def test_strict_env_guard_dumps_bad_control(tmp_path) -> None: class _NonfiniteControlEnv(_StubTorchEnv): def apply_action(self, actions: torch.Tensor, state: TorchEnvState) -> torch.Tensor: del state @@ -991,7 +978,6 @@ def apply_action(self, actions: torch.Tensor, state: TorchEnvState) -> torch.Ten NanGuard( NanGuardCfg( enabled=True, - defer_validation=True, check_interval=32, capture_interval=0, output_dir=str(tmp_path), diff --git a/tests/envs/locomotion/go2/test_manager_based_cfg.py b/tests/envs/locomotion/go2/test_manager_based_cfg.py index 00191b105..08b4e370a 100644 --- a/tests/envs/locomotion/go2/test_manager_based_cfg.py +++ b/tests/envs/locomotion/go2/test_manager_based_cfg.py @@ -304,6 +304,8 @@ def test_go2_flat_registry_executes_real_manager_runtime(backend: str, owner: st override = { **env_override, "events": {**env_override.get("events", {}), "pd_gains": None}, + # Real-runtime smoke test on the eager selected-row path on CPU. + "autoreset_cuda_graph": False, } env = registry.make( str(hydra_cfg.training.task_name), diff --git a/tests/envs/test_fixed_shape_autoreset.py b/tests/envs/test_fixed_shape_autoreset.py index 064964ac8..2061f94de 100644 --- a/tests/envs/test_fixed_shape_autoreset.py +++ b/tests/envs/test_fixed_shape_autoreset.py @@ -27,6 +27,7 @@ from tests.envs.test_manager_based_rl_env import _ResetBackend from unilab.base.entity import EntityCfg +from unilab.base.registry import apply_cfg_overrides from unilab.base.reset_selection import ( ResetSelection, assign_rows_, @@ -258,6 +259,22 @@ def _curriculum(env: ManagerBasedRlEnv, env_ids: Any) -> float: def _make_cfg(device: str, **overrides: Any) -> ManagerBasedRlEnvCfg: + """Build the canonical test env cfg. + + ``overrides`` first lands on declared ``ManagerBasedRlEnvCfg`` fields. + Names outside the collapsed public surface (``masked_autoreset``, + ``persistent_step_buffers``, ``defer_finite_validation``, + ``published_buffer_depth``, ``autoreset_graph_warmup``) attach as plain + attributes instead; ``TorchEnv`` reads them through its soft-``getattr`` + test-only injection protocol, so tests can construct intermediate chain + states that production Hydra configs can no longer express. + + The fixture pins ``autoreset_cuda_graph=False``: this file exercises the + fixed-shape chain through the test-only injection protocol and compares + it against the selected-row eager path, so the default-on public switch + (issue #2621) is pinned off here. Tests that exercise the public switch + itself set it through ``overrides``; its default is asserted separately. + """ cfg = ManagerBasedRlEnvCfg( scene=SceneCfg( model_file="fake.xml", @@ -268,6 +285,7 @@ def _make_cfg(device: str, **overrides: Any) -> ManagerBasedRlEnvCfg: max_episode_seconds=1.0, seed=11, collector_tensor_device=device, + autoreset_cuda_graph=False, observations={ "actor": ObservationGroupCfg( terms={"policy": ObservationTermCfg(func=_policy_obs, history_length=3)} @@ -285,7 +303,6 @@ def _make_cfg(device: str, **overrides: Any) -> ManagerBasedRlEnvCfg: metrics={"length": MetricsTermCfg(func=_metric, reduce="mean")}, policy_observation_group="actor", critic_observation_group="value", - defer_finite_validation=True, ) for name, value in overrides.items(): setattr(cfg, name, value) @@ -451,11 +468,31 @@ def test_circular_buffer_selection_reset_and_backfill_match_ids() -> None: # --------------------------------------------------------------------------- +def test_collapsed_switch_keys_fail_closed_as_env_overrides() -> None: + """Deleted chain keys raise a clear config error instead of being ignored.""" + cfg = _make_cfg("cpu") + for key in ( + "masked_autoreset", + "persistent_step_buffers", + "defer_finite_validation", + "published_buffer_depth", + "autoreset_graph_warmup", + ): + with pytest.raises(ValueError, match=f"has no attribute '{key}'"): + apply_cfg_overrides(cfg, {key: True}) + + def test_masked_autoreset_requires_deferred_validation() -> None: - with pytest.raises(ValueError, match="defer_finite_validation"): - _make_cfg("cpu", masked_autoreset=True, defer_finite_validation=False).validate() - with pytest.raises(ValueError, match="requires masked_autoreset"): - _make_cfg("cpu", autoreset_cuda_graph=True).validate() + with pytest.raises(ValueError, match="finite_validation_mode"): + _make_env("cpu", masked_autoreset=True, finite_validation_mode="strict") + with pytest.raises(ValueError, match="CUDA env device"): + _make_env("cpu", autoreset_cuda_graph=True) + + +def test_autoreset_cuda_graph_rejects_strict_validation() -> None: + cfg = _make_cfg("cuda", autoreset_cuda_graph=True, finite_validation_mode="strict") + with pytest.raises(ValueError, match="finite_validation_mode='deferred'"): + cfg.validate() def test_masked_autoreset_reports_every_unsupported_participant() -> None: @@ -583,16 +620,32 @@ def test_host_transitions_run_eagerly_outside_capture() -> None: cuda_only = pytest.mark.skipif(not CUDA, reason="requires CUDA") +@cuda_only +def test_autoreset_cuda_graph_implies_the_whole_fixed_shape_chain() -> None: + """Setting only the public switch enables the full chain (issue #2617).""" + env, _ = _make_env("cuda", autoreset_cuda_graph=True) + assert env.masked_autoreset + assert env.persistent_step_buffers + assert env.defer_finite_validation + assert env._reset_selection is not None + env.reset() + for action in _actions("cuda", 3): + env.step(action) + graph = env.autoreset_graph + assert graph is not None and graph.captured + assert graph.replays >= 1 + + @cuda_only def test_autoreset_graph_replay_matches_eager_masked_autoreset() -> None: actions = _actions("cuda", 30) eager_env, eager_backend = _make_env( "cuda", masked_autoreset=True, persistent_step_buffers=True ) + # The graph owner sets only the public switch; the masked/persistent + # middle states are its necessary consequences, not independent inputs. graph_env, graph_backend = _make_env( "cuda", - masked_autoreset=True, - persistent_step_buffers=True, autoreset_cuda_graph=True, autoreset_graph_warmup=2, ) @@ -676,3 +729,39 @@ def test_autoreset_graph_requires_cuda_and_backend_capability() -> None: persistent_step_buffers=True, autoreset_cuda_graph=True, ) + + +# --------------------------------------------------------------------------- +# Default-on flip (issue #2621) +# --------------------------------------------------------------------------- + + +def test_autoreset_cuda_graph_defaults_to_true() -> None: + """A task config that does not mention the switch runs the graph path.""" + assert ManagerBasedRlEnvCfg().autoreset_cuda_graph is True + + +def test_autoreset_cuda_graph_default_fails_closed_on_non_cuda_device() -> None: + """Default-on stays fail-closed off CUDA: startup raises, naming the switch. + + There is no silent eager fallback; the escape hatch is an explicit + ``autoreset_cuda_graph=false``. + """ + cfg = _make_cfg("cpu") + # A task YAML that does not set the switch keeps the dataclass default; + # deleting the instance attribute reads it back from the class default. + del cfg.autoreset_cuda_graph + assert cfg.autoreset_cuda_graph is True + backend = _TensorBackend(NUM_ENVS, "cpu") + with pytest.raises(ValueError, match="autoreset_cuda_graph requires a CUDA env device"): + ManagerBasedRlEnv(cfg, cast(SimBackend, backend), NUM_ENVS) + + +def test_autoreset_cuda_graph_explicit_false_keeps_the_selected_row_path() -> None: + """The explicit ``false`` escape hatch falls back to the eager lifecycle.""" + env, backend = _make_env("cpu", autoreset_cuda_graph=False) + assert not env.masked_autoreset + assert not env.persistent_step_buffers + assert env.autoreset_graph is None + trajectory, _ = _run(env, backend, _actions("cpu", 4)) + assert any(bool((snap["terminated"] | snap["truncated"]).any()) for snap in trajectory) diff --git a/tests/envs/test_go2_superdex.py b/tests/envs/test_go2_superdex.py index c4989027a..325451767 100644 --- a/tests/envs/test_go2_superdex.py +++ b/tests/envs/test_go2_superdex.py @@ -24,7 +24,11 @@ def _owner(backend: str) -> tuple[Any, dict[str, Any]]: GlobalHydra.instance().clear() with initialize_config_dir(config_dir=str(ROOT / "src/unilab/conf/ppo"), version_base="1.3"): owner = compose("config", overrides=[f"task=go2_joystick_flat/{backend}"]) - return owner, EnvConfigAdapter(owner, root_dir=ROOT).build_task_env_cfg_override() + override = EnvConfigAdapter(owner, root_dir=ROOT).build_task_env_cfg_override() + # These CPU-device tests pin the eager lifecycle; the default-on graph path + # fails closed earlier with its own device-capability blocker (#2621). + override["autoreset_cuda_graph"] = False + return owner, override def test_go2_superdex_preserves_mujoco_policy_contract() -> None: diff --git a/tests/envs/test_manager_based_rl_env.py b/tests/envs/test_manager_based_rl_env.py index 9c62f6539..583a2886d 100644 --- a/tests/envs/test_manager_based_rl_env.py +++ b/tests/envs/test_manager_based_rl_env.py @@ -1301,6 +1301,9 @@ def _make_cfg( ctrl_dt=0.01 * sim_substeps, max_episode_seconds=1.0, seed=7, + # This file exercises the eager selected-row lifecycle on a fake + # CPU backend; pin the default-on public switch (issue #2621) off. + autoreset_cuda_graph=False, observations=observations, actions={"drive": _DriveCfg(entity_name="robot")} if actions is None else actions, rewards={"track": RewardTermCfg(func=_reward, weight=1.0)} if rewards is None else rewards, @@ -1404,7 +1407,6 @@ def test_deferred_episode_metrics_flush_at_fixed_step_without_reset() -> None: metrics={"length": MetricsTermCfg(func=_metric, reduce="mean")}, ) cfg.terminations = {"failure": TerminationTermCfg(func=_failure)} - cfg.defer_finite_validation = True cfg.finite_validation_interval = 4 env, _ = _make_env(cfg) env.reset() @@ -1429,7 +1431,6 @@ def test_deferred_episode_metrics_flush_at_fixed_step_without_reset() -> None: def test_packed_reset_validation_mode_follows_defer_finite_validation() -> None: cfg = _make_cfg(include_optional_managers=False) - cfg.defer_finite_validation = True env, _backend = _make_env(cfg) try: env._reset_state.declare_packed_reset_device(env.device) @@ -1442,7 +1443,7 @@ def test_packed_reset_validation_mode_follows_defer_finite_validation() -> None: env.close() eager_cfg = _make_cfg(include_optional_managers=False) - eager_cfg.defer_finite_validation = False + eager_cfg.finite_validation_mode = "strict" eager_env, _ = _make_env(eager_cfg) try: eager_env._reset_state.declare_packed_reset_device(eager_env.device) @@ -1455,7 +1456,6 @@ def test_packed_reset_validation_mode_follows_defer_finite_validation() -> None: def test_packed_reset_deferred_verdict_raises_at_poll_boundary() -> None: cfg = _make_cfg(include_optional_managers=False) - cfg.defer_finite_validation = True cfg.finite_validation_interval = 4 env, _backend = _make_env(cfg) try: @@ -1709,6 +1709,8 @@ def test_mujoco_host_bridge_packed_reset_lifecycle( sim_dt=0.01, ctrl_dt=0.02, max_episode_seconds=0.04, + # Host-bridge MuJoCo observation test on the eager selected-row path. + autoreset_cuda_graph=False, observations={ "actor": ObservationGroupCfg( terms={"joint": ObservationTermCfg(func=mdp.joint_pos_rel)} @@ -1779,6 +1781,8 @@ def build(pattern: mdp.RayPatternCfg): sim_dt=0.01, ctrl_dt=0.02, max_episode_seconds=0.04, + # Host-bridge ray-observation test on the eager selected-row path. + autoreset_cuda_graph=False, observations={ "actor": ObservationGroupCfg( terms={ @@ -2461,6 +2465,8 @@ def test_real_mujoco_backend_runs_packed_reset_lifecycle() -> None: sim_dt=0.01, ctrl_dt=0.02, max_episode_seconds=1.0, + # Real-backend packed-reset test on the eager selected-row path. + autoreset_cuda_graph=False, observations={ "actor": ObservationGroupCfg( terms={"state": ObservationTermCfg(func=_episode_step_observation)} @@ -2539,6 +2545,8 @@ def test_real_mujoco_default_state_applies_keyframe_baseline( sim_dt=0.01, ctrl_dt=0.02, max_episode_seconds=1.0, + # Host-state playback test on the eager selected-row path. + autoreset_cuda_graph=False, observations={ "actor": ObservationGroupCfg( terms={"state": ObservationTermCfg(func=_episode_step_observation)} diff --git a/tests/envs/test_multi_entity_consumer.py b/tests/envs/test_multi_entity_consumer.py index cec4d8c96..5ae1a0911 100644 --- a/tests/envs/test_multi_entity_consumer.py +++ b/tests/envs/test_multi_entity_consumer.py @@ -183,6 +183,8 @@ def build_fixture_cfg(*, passive: bool, num_envs: int = 2, mirror: bool | None = sim_dt=0.001, ctrl_dt=0.001, max_episode_seconds=1.0, + # Registry factory tests run the eager selected-row path on CPU. + autoreset_cuda_graph=False, observations={ "policy": ObservationGroupCfg(terms={"state": ObservationTermCfg(func=observations)}) }, diff --git a/tests/managers/test_isaac_lab_migration_fixture.py b/tests/managers/test_isaac_lab_migration_fixture.py index 38807cc05..3f7d2e782 100644 --- a/tests/managers/test_isaac_lab_migration_fixture.py +++ b/tests/managers/test_isaac_lab_migration_fixture.py @@ -88,6 +88,8 @@ def test_fixture_real_mujoco_manager_runs_packed_reset_lifecycle() -> None: reason="unisim-core MuJoCo adapter (mjbatch build) not available", ) _, _, override = _materialize_fixture() + # Packed-reset lifecycle test on the eager selected-row path on CPU. + override["autoreset_cuda_graph"] = False env = registry.make( FIXTURE_ENV_NAME, sim_backend="mujoco", @@ -111,6 +113,8 @@ def test_fixture_missing_actuator_fails_during_cold_path_binding() -> None: ) _, _, override = _materialize_fixture() override["actions"]["joint_effort"]["actuator_names"] = ["missing_actuator"] + # Keep the cold-path assertion reachable: run the eager path on CPU. + override["autoreset_cuda_graph"] = False with pytest.raises( ValueError, diff --git a/tests/managers/test_mjlab_migration_fixture.py b/tests/managers/test_mjlab_migration_fixture.py index ba709f2e0..802814a93 100644 --- a/tests/managers/test_mjlab_migration_fixture.py +++ b/tests/managers/test_mjlab_migration_fixture.py @@ -75,6 +75,8 @@ def test_mjlab_fixture_real_mujoco_manager_runs_packed_reset_lifecycle() -> None reason="unisim-core MuJoCo adapter (mjbatch build) not available", ) _, override = _materialize() + # These packed-reset lifecycle tests run the eager selected-row path on CPU. + override["autoreset_cuda_graph"] = False env = registry.make( FIXTURE_ENV_NAME, sim_backend="mujoco", @@ -98,6 +100,8 @@ def test_mjlab_fixture_missing_actuator_fails_on_cold_path() -> None: ) _, override = _materialize() override["actions"]["effort"]["actuator_names"] = ["missing_actuator"] + # Keep the cold-path assertion reachable: run the eager path on CPU. + override["autoreset_cuda_graph"] = False with pytest.raises(ValueError, match="regular expressions matched.*missing_actuator"): registry.make( FIXTURE_ENV_NAME, diff --git a/tests/managers/test_stage_curriculum_demo.py b/tests/managers/test_stage_curriculum_demo.py index cd2d9e1f1..1db04bf1e 100644 --- a/tests/managers/test_stage_curriculum_demo.py +++ b/tests/managers/test_stage_curriculum_demo.py @@ -67,6 +67,8 @@ def test_stage_curriculum_demo_real_backend_runs_packed_reset_lifecycle() -> Non reason="unisim-core MuJoCo adapter (mjbatch build) not available", ) _, override = _materialize() + # These packed-reset lifecycle tests run the eager selected-row path on CPU. + override["autoreset_cuda_graph"] = False env = registry.make( FIXTURE_ENV_NAME, sim_backend="mujoco", @@ -93,6 +95,8 @@ def test_stage_curriculum_demo_owner_invalid_stages_fail_closed() -> None: {"step": 4, "weight": 0.5}, {"step": 2, "weight": 0.25}, ] + # Keep the fail-closed assertion reachable: run the eager path on CPU. + override["autoreset_cuda_graph"] = False with pytest.raises(ValueError, match="nondecreasing"): registry.make( FIXTURE_ENV_NAME, diff --git a/tests/rl/runtime/ipc/test_offpolicy_completion_integration.py b/tests/rl/runtime/ipc/test_offpolicy_completion_integration.py index 6da853769..92c7960f6 100644 --- a/tests/rl/runtime/ipc/test_offpolicy_completion_integration.py +++ b/tests/rl/runtime/ipc/test_offpolicy_completion_integration.py @@ -37,6 +37,8 @@ def test_eager_mujoco_replay_rows_own_their_tick_across_autoreset() -> None: with initialize_config_dir(config_dir=str(CONF_DIR / "ppo"), version_base="1.3"): cfg = compose("config", overrides=["task=go2_joystick_flat/mujoco"]) override = EnvConfigAdapter(cfg, root_dir=ROOT_DIR).build_task_env_cfg_override() + # The replay integration drives the eager selected-row lifecycle on CPU. + override["autoreset_cuda_graph"] = False env = registry.make( str(cfg.training.task_name), sim_backend="mujoco", diff --git a/tests/tasks/test_g1_motion_fixed_shape.py b/tests/tasks/test_g1_motion_fixed_shape.py index c8099d3f8..f926a98e0 100644 --- a/tests/tasks/test_g1_motion_fixed_shape.py +++ b/tests/tasks/test_g1_motion_fixed_shape.py @@ -72,9 +72,9 @@ def test_g1_motion_tracking_captures_and_replays_autoreset_without_host_sync() - pytest.skip("CUDA graph capture requires a CUDA device") env = _make_env( 64, - mjwarp_defer_tensor_validation=True, - masked_autoreset=True, - persistent_step_buffers=True, + # The public switch implies the masked/persistent chain, and the + # deferred validation mode derives the backend defer option + # (issue #2617, #2618). autoreset_cuda_graph=True, # Poll explicitly below so every step is checked for host syncs. finite_validation_interval=10**6, diff --git a/tests/training/test_tensor_diagnostics.py b/tests/training/test_tensor_diagnostics.py index 87a2995ab..f4aacf58f 100644 --- a/tests/training/test_tensor_diagnostics.py +++ b/tests/training/test_tensor_diagnostics.py @@ -10,7 +10,7 @@ import torch from omegaconf import OmegaConf -from unilab.training.run import finite_validation_env_overrides, resolve_nan_guard_cfg +from unilab.training.run import reject_removed_validation_keys, resolve_nan_guard_cfg from unilab.training.tensor_diagnostics import NanGuard, NanGuardCfg @@ -57,7 +57,8 @@ def test_detection_unions_bad_rows_across_groups_and_reward() -> None: def test_deferred_verdict_accumulates_mixed_rows_until_poll(tmp_path: Path) -> None: - guard = _guard(tmp_path, defer_validation=True, check_interval=4) + guard = _guard(tmp_path, check_interval=4) + guard.set_defer_validation(True) obs = {"obs": torch.tensor([[1.0, 2.0], [3.0, float("nan")], [4.0, 5.0]])} assert guard.check(obs, torch.zeros(3), step=1) is None assert guard.check_ctrl(torch.tensor([[0.0], [1.0], [float("inf")]]), step=2) is None @@ -77,7 +78,8 @@ def test_deferred_verdict_accumulates_mixed_rows_until_poll(tmp_path: Path) -> N def test_explicit_non_deferred_check_ignores_configured_poll_interval( tmp_path: Path, check_interval: int ) -> None: - guard = _guard(tmp_path, defer_validation=True, check_interval=check_interval) + guard = _guard(tmp_path, check_interval=check_interval) + guard.set_defer_validation(True) obs = {"obs": torch.tensor([[0.0], [float("nan")], [0.0]])} obs_ids = guard.check(obs, torch.zeros(3), step=1, defer=False) @@ -170,36 +172,33 @@ def test_configuration_and_shapes_fail_closed(tmp_path: Path) -> None: guard.check({"obs": torch.zeros((3, 2), dtype=torch.int32)}, torch.zeros(3)) -def test_training_routes_guard_and_shared_finite_boundary() -> None: +def test_training_routes_guard_and_rejects_removed_validation_keys() -> None: cfg = OmegaConf.create( { "training": { - "defer_finite_validation": True, - "finite_validation_interval": 32, "nan_guard": { "enabled": True, "capture_interval": 32, "check_interval": 32, - "defer_validation": True, }, } } ) - assert finite_validation_env_overrides(cfg) == { - "defer_finite_validation": True, - "finite_validation_interval": 32, - } guard_cfg = resolve_nan_guard_cfg(cfg.training) assert guard_cfg is not None - assert (guard_cfg.capture_interval, guard_cfg.check_interval, guard_cfg.defer_validation) == ( - 32, - 32, - True, - ) - - cfg.training.finite_validation_interval = 0 - with pytest.raises(ValueError, match="positive integer"): - finite_validation_env_overrides(cfg) + assert (guard_cfg.capture_interval, guard_cfg.check_interval) == (32, 32) + reject_removed_validation_keys(cfg) + + cfg.training.finite_validation_interval = 32 + with pytest.raises(ValueError, match="training.finite_validation_interval was removed"): + reject_removed_validation_keys(cfg) + del cfg.training.finite_validation_interval + + cfg.training.nan_guard.defer_validation = True + with pytest.raises(ValueError, match="training.nan_guard.defer_validation was removed"): + reject_removed_validation_keys(cfg) + with pytest.raises(ValueError, match="training.nan_guard.defer_validation was removed"): + resolve_nan_guard_cfg(cfg.training) def test_reset_verdict_allows_inference_tensor_latches(tmp_path: Path) -> None: