Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/sphinx/source/adr/ADR-0000-index.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ orphan: true
| [ADR-0020 FlashSAC DP Graph Ownership And Wire Precision](ADR-0020-flashsac-dp-graph-and-wire-precision.md) | RL algorithm / distributed runtime | Proposed |
| [ADR-0021 Genesis Device-Resident DR Lifecycle](ADR-0021-genesis-device-resident-dr-lifecycle.md) | Backend DR / Manager reset plane | Accepted |
| [ADR-0022 UniSim Owns Math Rotation Primitives](ADR-0022-unisim-owns-math-rotation-primitives.md) | Math primitives / package ownership | Proposed |
| [ADR-0023 Task Sim Owner Inherits Same-Directory Base Only](ADR-0023-task-sim-owner-inherits-same-directory-base-only.md) | Config owner / task YAML inheritance | Proposed |
| [ADR-0231 Integration-only K=1 MJWarp Collector CUDA Graph](ADR-0231-k1-mjwarp-collector-cuda-graph-integration-only.md) | Collector CUDA graph / historical pre-registration | Rejected / no-go at M2 |
| [ADR-0238 State-Update-Only CUDA Graph Boundary](ADR-0238-state-update-only-cuda-graph-boundary.md) | Manager CUDA graph / historical pre-registration | Rejected / no-go |

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
---
orphan: true
---

# ADR-0023 Task Sim Owner Inherits Same-Directory Base Only

语言: 简体中文

- Status: Proposed
- Date: 2026-11-18
- Owners: Config / Env maintainers
- Supersedes: None
- Superseded by: None

## Context

`src/unilab/conf/<algo>/task/<task>/` 下的仿真 owner YAML(backend leaf)历史上
出现了跨后端继承:`g1_motion_tracking` 的 motrix/mjwarp/genesis/newton leaf 直接
继承同目录的 `mujoco.yaml`,`go2_joystick_flat/superdex.yaml` 同样继承
`mujoco.yaml`。这带来两个问题:

- 隐式耦合:leaf 的组合结果依赖另一个后端的 owner,修改 MuJoCo owner 的
reward/env/algo 会静默改变 Motrix/MJWarp/Genesis 的训练行为,review 无法从
leaf 文件本身看出其真实契约。
- 身份混淆:mujoco owner 同时承担"任务 canonical contract"与"MuJoCo 后端身份"
两个角色,违背 ADR-0003 的 task owner 身份语义。

2026-11-18 的清理确认了该问题的范围:sac/flashsac/appo/ppo 四个算法树的
`g1_motion_tracking` 与 ppo `go2_joystick_flat` 共 10 个 leaf 违规,其中
appo/ppo 的 `g1_motion_tracking` 目录连 `base.yaml` 都不存在,mujoco owner
被迫充当 base。

## Decision

- 每个 `<algo>/task/<task>/` 目录必须包含一个 `base.yaml`,持有该任务对该算法的
canonical contract:scene/observations/actions/commands/terminations/reward,
canonical DR `env.events`,以及后端共享的 training budget(`training.task_name`、
`algo` 预算)。
- 目录内每个仿真 backend leaf(`mujoco.yaml`、`mjwarp.yaml`、`genesis.yaml`、
`motrix.yaml`、`newton.yaml`、`superdex.yaml`、`drake.yaml` 等)的 Hydra
`defaults:` 只允许列出 `/task/<task>/base` 与 `_self_`,禁止继承任何其他
backend leaf。leaf 直接持有身份字段(`training.task_name`、
`training.sim_backend`,见 ADR-0003)、渲染/设备 placement,以及显式支持
或禁用的 DR term(如 `events.push_robot: null`)。
- 若某后端确有独立的调参需求(如 superdex 的 `max_iterations`、观测归一化),
必须在该 leaf 内显式写出完整数值,不允许通过继承其他后端 owner 获得。
- `base.yaml` 自身不是可训练入口(不持有 `training.sim_backend`),仅作为
contract 声明与 leaf 的 compose 基线。

## Stable Contracts

- `src/unilab/conf/<algo>/task/<task>/<backend>.yaml` 的 `defaults:` 恰好包含
`/task/<task>/base` 与 `_self_`;该结构由
`tests/config/test_config_system.py` 的继承结构测试全量守护。
- backend leaf 的组合结果必须能从 `base.yaml + leaf 文件本身` 完整推出;
review 时 leaf 文件即其完整 backend 契约。
- 后端间共享的 DR/奖励定义收敛在 `base.yaml`;单个后端不支持某 term 时,
在该后端 leaf 中显式置 `null` 或收窄参数,禁止"借道"其他后端 owner。
- 跨算法对齐(如 flashsac 与 sac 的 `g1_motion_tracking` 任务契约)通过使两边
`base.yaml` 一致来表达,而不是通过跨树继承。

## Alternatives Considered

- **保留跨后端继承、仅加注释约束:** 拒绝。继承链使行为变更不可见,正是本次
清理要消除的问题;注释无法被测试守护。
- **允许继承同任务任意 leaf、只禁止跨任务继承:** 拒绝。同任务跨后端继承同样
让 MuJoCo owner 的修改静默传播到其他后端,违背"leaf 文件即完整契约"目标。
- **用 Hydra config group(`task/g1_motion_tracking` 组默认值)代替显式
defaults:** 拒绝。组默认值是隐式路由,review 时同样看不到 leaf 的 compose
基线;显式 `defaults: [/task/<task>/base, _self_]` 一目了然。

## Consequences

- 后端 owner 之间出现少量显式数值重复(如 superdex 携带观测归一化与学习率)。
这是有意的:重复可见优于继承隐式。
- 修改 `base.yaml` 等同于修改该任务所有后端的契约,必须按 ADR-0003 的
owner 变更流程评审;修改单个 leaf 不再影响其他后端。
- 新增 backend leaf 时必须先确认同目录 `base.yaml` 存在;目录内没有
`base.yaml` 时禁止新增 leaf。
- 2026-11-18 的迁移以"组合结果不变"为验收标准:除
flashsac `g1_motion_tracking` 按本仓库决策向 sac 契约对齐(补齐 canonical DR、
`geom_names`、actor 有偏关节观测)外,所有受影响 owner 的 Hydra 组合输出
与迁移前逐键一致。

## Evidence In Repo

- owner 配置树:`src/unilab/conf/{sac,flashsac,appo,ppo}/task/<task>/`。
- 对齐示例:flashsac `g1_motion_tracking` 的 `base.yaml` 与 sac 同任务
`base.yaml` 契约一致(scene/observations/commands/reward/terminations/DR)。
- 继承结构守护测试:`tests/config/test_config_system.py`。
- 组合等价与任务契约测试:`tests/config/test_g1_motion_tracking_owners.py`。

## Related Documents

- {doc}`ADR Index </adr/ADR-0000-index>`
- {doc}`ADR-0003 Task Owner And Config Compose Contract </adr/ADR-0003-task-owner-and-config-compose-contract>`
- {doc}`RL Infrastructure 开发标准 </zh_CN/4-developer_guide/0-index>`
Original file line number Diff line number Diff line change
Expand Up @@ -47,9 +47,12 @@ the MJWarp owner inherits the MuJoCo FlashSAC owner and resolves to:
| Motion | `motions/g1/dance1_subject2_part.npz` |

The MuJoCo owner uses the same 2048-environment training budget, but publishes
collector metrics every step. The MJWarp task is also a DR-free lifecycle
baseline: its selected-row tensor reset does not negotiate reset randomization
or interval wrenches.
collector metrics every step. The MJWarp owner inherits the same canonical DR
(reset-mode model-field randomization plus the linear-velocity interval push)
and only zeroes the angular-velocity kicks, because its selected-row tensor
reset does not negotiate the interval angular root-velocity capability. The
Newton owner does not support reset randomization and sets the canonical DR
terms to `null` explicitly.

At the current generated evidence grade, FlashSAC /
`g1_motion_tracking` / MJWarp is `Configured`. A successful local soak does not
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,10 @@ env.reset_selection_blockers() # [] when the task can enable masked_autoreset
```

Then convert, replace or remove each named term. Reset-randomization events are
common blockers; the G1 motion-tracking FlashSAC owner
(`src/unilab/conf/flashsac/task/g1_motion_tracking/base.yaml`) is a DR-free
baseline for this reason.
common blockers; when a backend cannot negotiate them, the task owner must
disable those terms explicitly. The G1 motion-tracking owners keep the
canonical DR contract in
`src/unilab/conf/<algo>/task/g1_motion_tracking/base.yaml`, and backends without
reset-randomization support (for example the FlashSAC Newton owner) set the
unsupported event terms to `null` in their own leaf instead of forking the
contract.
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,10 @@ MJWarp owner 继承 FlashSAC MuJoCo owner,并解析为:
| Motion | `motions/g1/dance1_subject2_part.npz` |

MuJoCo owner 使用同样的 2048 环境训练预算,但每步发布 collector metric。
MJWarp task 也是 DR-free lifecycle baseline:其 selected-row tensor reset
不协商 reset randomization 或 interval wrench。
MJWarp owner 继承同样的 canonical DR(reset 模式模型字段随机化 + 线速度
interval push),仅将角速度推扰收窄为零,因为其 selected-row tensor reset
不协商 interval angular root-velocity。Newton owner 不支持 reset 随机化,
将 canonical DR 显式置 `null`。

在当前生成证据等级中,FlashSAC / `g1_motion_tracking` / MJWarp 是
`Configured`。本地 soak 成功不会自动把它提升为 `Benchmarked` 或
Expand Down
236 changes: 236 additions & 0 deletions src/unilab/conf/appo/task/g1_motion_tracking/base.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,236 @@
# @package _global_
# Canonical G1 WBT task declaration for APPO owners. Backend owners inherit
# this contract and override only backend identity; sim owners never inherit
# each other (ADR-0023).
training:
task_name: G1MotionTracking
play_steps: 1000

algo:
num_envs: 1024
max_iterations: 5000
save_interval: 500
algorithm:
adaptive_kl_factor: 2.0
adaptive_lr_factor: 1.5

env:
scene:
model_file: src/unilab/assets/robots/g1/scene_flat.xml
default_keyframe_name: stand
entities:
robot:
root_body_name: pelvis
joint_names: &g1_joints
- left_hip_pitch_joint
- left_hip_roll_joint
- left_hip_yaw_joint
- left_knee_joint
- left_ankle_pitch_joint
- left_ankle_roll_joint
- right_hip_pitch_joint
- right_hip_roll_joint
- right_hip_yaw_joint
- right_knee_joint
- right_ankle_pitch_joint
- right_ankle_roll_joint
- waist_yaw_joint
- waist_roll_joint
- waist_pitch_joint
- left_shoulder_pitch_joint
- left_shoulder_roll_joint
- left_shoulder_yaw_joint
- left_elbow_joint
- left_wrist_roll_joint
- left_wrist_pitch_joint
- left_wrist_yaw_joint
- right_shoulder_pitch_joint
- right_shoulder_roll_joint
- right_shoulder_yaw_joint
- right_elbow_joint
- right_wrist_roll_joint
- right_wrist_pitch_joint
- right_wrist_yaw_joint
actuator_names: *g1_joints
body_names: &tracked_bodies
- pelvis
- left_hip_roll_link
- left_knee_link
- left_ankle_roll_link
- right_hip_roll_link
- right_knee_link
- right_ankle_roll_link
- torso_link
- left_shoulder_roll_link
- left_elbow_link
- left_wrist_yaw_link
- right_shoulder_roll_link
- right_elbow_link
- right_wrist_yaw_link
geom_names:
- left_foot1_collision
- left_foot2_collision
- left_foot3_collision
- left_foot4_collision
- left_foot5_collision
- left_foot6_collision
- left_foot7_collision
- right_foot1_collision
- right_foot2_collision
- right_foot3_collision
- right_foot4_collision
- right_foot5_collision
- right_foot6_collision
- right_foot7_collision
sim_dt: 0.006666666666666667
ctrl_dt: 0.02
max_episode_seconds: 10.0
observations:
actor:
terms:
command: &command_obs
func: unilab.envs.mdp.generated_commands
params: {command_name: motion}
motion_anchor_pos_b: &anchor_pos_obs
func: unilab.tasks.motion_tracking.common.manager_terms.motion_anchor_pos_b
params: {command_name: motion}
motion_anchor_ori_b: &anchor_ori_obs
func: unilab.tasks.motion_tracking.common.manager_terms.motion_anchor_ori_b
params: {command_name: motion}
base_lin_vel: &base_lin_vel_obs
func: unilab.envs.mdp.builtin_sensor
params: {sensor_name: pelvis_local_linvel, entity_name: robot}
base_ang_vel: &base_ang_vel_obs
func: unilab.envs.mdp.builtin_sensor
params: {sensor_name: torso_gyro, entity_name: robot}
joint_pos: &joint_pos_obs
func: unilab.tasks.motion_tracking.common.manager_terms.motion_joint_pos_rel
params: {command_name: motion}
joint_vel: &joint_vel_obs
func: unilab.envs.mdp.joint_vel_rel
actions: &actions_obs
func: unilab.envs.mdp.last_action
critic:
terms:
command: *command_obs
motion_anchor_pos_b: *anchor_pos_obs
motion_anchor_ori_b: *anchor_ori_obs
base_lin_vel: *base_lin_vel_obs
base_ang_vel: *base_ang_vel_obs
joint_pos: *joint_pos_obs
joint_vel: *joint_vel_obs
actions: *actions_obs
body_pos:
func: unilab.tasks.motion_tracking.common.manager_terms.robot_body_pos_b
params: {command_name: motion}
body_ori:
func: unilab.tasks.motion_tracking.common.manager_terms.robot_body_ori_b
params: {command_name: motion}
actions:
joint_pos:
_target_: unilab.tasks.motion_tracking.common.manager_terms.MotionJointPositionActionCfg
entity_name: robot
actuator_names: [".*"]
scale: 0.25
use_default_offset: true
command_name: motion
commands:
motion:
_target_: unilab.tasks.motion_tracking.common.manager_terms.MotionCommandCfg
entity_name: robot
resampling_time_range: [1.0e9, 1.0e9]
params:
motion_file: motions/g1/dance1_subject2_part.npz
anchor_body_name: torso_link
body_names: *tracked_bodies
sampling_mode: adaptive
sampling_start_ratio: 0.0
truncate_on_clip_end: false
pose_range:
x: [-0.05, 0.05]
y: [-0.05, 0.05]
z: [-0.01, 0.01]
roll: [-0.1, 0.1]
pitch: [-0.1, 0.1]
yaw: [-0.2, 0.2]
velocity_range:
x: [-0.5, 0.5]
y: [-0.5, 0.5]
z: [-0.2, 0.2]
roll: [-0.52, 0.52]
pitch: [-0.52, 0.52]
yaw: [-0.78, 0.78]
joint_position_range: [-0.1, 0.1]
joint_default_position_range: [0.0, 0.0]
terminations:
time_out:
func: unilab.envs.mdp.time_out
time_out: true
anchor_pos:
func: unilab.tasks.motion_tracking.common.manager_terms.BadAnchorPosZOnly
params: {command_name: motion, threshold: 0.25}
anchor_ori:
func: unilab.tasks.motion_tracking.common.manager_terms.bad_anchor_ori
params:
command_name: motion
threshold: 0.8
asset_cfg:
_target_: unilab.managers.SceneEntityCfg
name: robot
ee_body_pos:
func: unilab.tasks.motion_tracking.common.manager_terms.BadMotionBodyPosZOnly
params:
command_name: motion
threshold: 0.25
body_names:
- left_ankle_roll_link
- right_ankle_roll_link
- left_wrist_yaw_link
- right_wrist_yaw_link
policy_observation_group: actor
critic_observation_group: critic

reward:
motion_global_root_pos:
func: unilab.tasks.motion_tracking.common.manager_terms.motion_global_anchor_position_error_exp
weight: 0.5
params: {command_name: motion, std: 0.3}
motion_global_root_ori:
func: unilab.tasks.motion_tracking.common.manager_terms.motion_global_anchor_orientation_error_exp
weight: 0.5
params: {command_name: motion, std: 0.4}
motion_body_pos:
func: unilab.tasks.motion_tracking.common.manager_terms.MotionRelativeBodyPositionErrorExp
weight: 1.0
params: {command_name: motion, std: 0.3}
motion_body_ori:
func: unilab.tasks.motion_tracking.common.manager_terms.MotionRelativeBodyOrientationErrorExp
weight: 1.0
params: {command_name: motion, std: 0.4}
motion_body_lin_vel:
func: unilab.tasks.motion_tracking.common.manager_terms.MotionGlobalBodyLinearVelocityErrorExp
weight: 1.0
params: {command_name: motion, std: 1.0}
motion_body_ang_vel:
func: unilab.tasks.motion_tracking.common.manager_terms.MotionGlobalBodyAngularVelocityErrorExp
weight: 1.0
params: {command_name: motion, std: 3.14}
motion_joint_pos:
func: unilab.tasks.motion_tracking.common.manager_terms.motion_joint_position_error_exp
weight: 0.0
params: {command_name: motion, std: 0.2}
motion_joint_vel:
func: unilab.tasks.motion_tracking.common.manager_terms.motion_joint_velocity_error_exp
weight: 0.0
params: {command_name: motion, std: 1.0}
action_rate_l2:
func: unilab.envs.mdp.action_rate_l2
weight: -0.1
joint_limit:
func: unilab.tasks.motion_tracking.common.manager_terms.joint_pos_limits
weight: -10.0
params:
asset_cfg:
_target_: unilab.managers.SceneEntityCfg
name: robot
joint_names: ".*"
2 changes: 1 addition & 1 deletion src/unilab/conf/appo/task/g1_motion_tracking/motrix.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# @package _global_
defaults:
- /task/g1_motion_tracking/mujoco
- /task/g1_motion_tracking/base
- _self_

training:
Expand Down
Loading
Loading