From 9d1bbaf7b45ae0bb0f0035a28ffe65d67e2a18f0 Mon Sep 17 00:00:00 2001 From: TATP-233 Date: Wed, 18 Nov 2026 20:37:04 +0800 Subject: [PATCH] feat(conf): align flashsac g1_motion_tracking with sac and enforce same-dir base inheritance (ADR-0023) - Align the flashsac g1_motion_tracking owner with the SAC contract: canonical DR events, foot geom_names, and the biased actor joint observation; training budget already matched. The Newton owner keeps its historical DR-free behavior with events explicitly nulled. - Every task sim owner leaf now inherits only its same-directory base.yaml; cross-backend inheritance removed (sac/flashsac/appo/ppo g1_motion_tracking, ppo go2_joystick_flat/superdex). Shared DR and training budgets move into base.yaml; leaves carry their full task_name/sim_backend identity. - Add ADR-0023 and a structural guard test; composed configs verified key-identical before/after except the flashsac alignment itself. Training behavior: unchanged except flashsac g1_motion_tracking aligning to the SAC owner. Affects MuJoCo, MJWarp, Motrix, Genesis, and Newton owner YAMLs for g1_motion_tracking on Linux. Ref #2613 --- docs/sphinx/source/adr/ADR-0000-index.md | 1 + ...owner-inherits-same-directory-base-only.md | 97 +++++++ .../1-training/7-tensor_runtime_production.md | 9 +- .../1-training/8-sync_free_autoreset.md | 10 +- .../1-training/7-tensor_runtime_production.md | 6 +- .../appo/task/g1_motion_tracking/base.yaml | 236 +++++++++++++++++ .../appo/task/g1_motion_tracking/motrix.yaml | 2 +- .../appo/task/g1_motion_tracking/mujoco.yaml | 235 +---------------- .../task/g1_motion_tracking/base.yaml | 101 +++++++- .../task/g1_motion_tracking/genesis.yaml | 37 ++- .../task/g1_motion_tracking/mjwarp.yaml | 15 +- .../task/g1_motion_tracking/motrix.yaml | 11 +- .../task/g1_motion_tracking/mujoco.yaml | 16 +- .../task/g1_motion_tracking/newton.yaml | 12 +- .../ppo/task/g1_motion_tracking/base.yaml | 242 ++++++++++++++++++ .../ppo/task/g1_motion_tracking/motrix.yaml | 2 +- .../ppo/task/g1_motion_tracking/mujoco.yaml | 241 +---------------- .../ppo/task/go2_joystick_flat/superdex.yaml | 21 +- .../sac/task/g1_motion_tracking/base.yaml | 71 ++++- .../sac/task/g1_motion_tracking/genesis.yaml | 18 +- .../sac/task/g1_motion_tracking/mjwarp.yaml | 5 +- .../sac/task/g1_motion_tracking/motrix.yaml | 3 +- .../sac/task/g1_motion_tracking/mujoco.yaml | 68 +---- tests/config/test_config_system.py | 26 ++ .../config/test_g1_motion_tracking_owners.py | 72 ++++-- 25 files changed, 946 insertions(+), 611 deletions(-) create mode 100644 docs/sphinx/source/adr/ADR-0023-task-sim-owner-inherits-same-directory-base-only.md create mode 100644 src/unilab/conf/appo/task/g1_motion_tracking/base.yaml create mode 100644 src/unilab/conf/ppo/task/g1_motion_tracking/base.yaml diff --git a/docs/sphinx/source/adr/ADR-0000-index.md b/docs/sphinx/source/adr/ADR-0000-index.md index d71475ace..cf5123864 100644 --- a/docs/sphinx/source/adr/ADR-0000-index.md +++ b/docs/sphinx/source/adr/ADR-0000-index.md @@ -34,6 +34,7 @@ orphan: true | [ADR-0020 FlashSAC DP Graph Ownership And Wire Precision](ADR-0020-flashsac-dp-graph-and-wire-precision.md) | RL algorithm / distributed runtime | Proposed | | [ADR-0021 Genesis Device-Resident DR Lifecycle](ADR-0021-genesis-device-resident-dr-lifecycle.md) | Backend DR / Manager reset plane | Accepted | | [ADR-0022 UniSim Owns Math Rotation Primitives](ADR-0022-unisim-owns-math-rotation-primitives.md) | Math primitives / package ownership | Proposed | +| [ADR-0023 Task Sim Owner Inherits Same-Directory Base Only](ADR-0023-task-sim-owner-inherits-same-directory-base-only.md) | Config owner / task YAML inheritance | Proposed | | [ADR-0231 Integration-only K=1 MJWarp Collector CUDA Graph](ADR-0231-k1-mjwarp-collector-cuda-graph-integration-only.md) | Collector CUDA graph / historical pre-registration | Rejected / no-go at M2 | | [ADR-0238 State-Update-Only CUDA Graph Boundary](ADR-0238-state-update-only-cuda-graph-boundary.md) | Manager CUDA graph / historical pre-registration | Rejected / no-go | diff --git a/docs/sphinx/source/adr/ADR-0023-task-sim-owner-inherits-same-directory-base-only.md b/docs/sphinx/source/adr/ADR-0023-task-sim-owner-inherits-same-directory-base-only.md new file mode 100644 index 000000000..d2d766ccd --- /dev/null +++ b/docs/sphinx/source/adr/ADR-0023-task-sim-owner-inherits-same-directory-base-only.md @@ -0,0 +1,97 @@ +--- +orphan: true +--- + +# ADR-0023 Task Sim Owner Inherits Same-Directory Base Only + +语言: 简体中文 + +- Status: Proposed +- Date: 2026-11-18 +- Owners: Config / Env maintainers +- Supersedes: None +- Superseded by: None + +## Context + +`src/unilab/conf//task//` 下的仿真 owner YAML(backend leaf)历史上 +出现了跨后端继承:`g1_motion_tracking` 的 motrix/mjwarp/genesis/newton leaf 直接 +继承同目录的 `mujoco.yaml`,`go2_joystick_flat/superdex.yaml` 同样继承 +`mujoco.yaml`。这带来两个问题: + +- 隐式耦合:leaf 的组合结果依赖另一个后端的 owner,修改 MuJoCo owner 的 + reward/env/algo 会静默改变 Motrix/MJWarp/Genesis 的训练行为,review 无法从 + leaf 文件本身看出其真实契约。 +- 身份混淆:mujoco owner 同时承担"任务 canonical contract"与"MuJoCo 后端身份" + 两个角色,违背 ADR-0003 的 task owner 身份语义。 + +2026-11-18 的清理确认了该问题的范围:sac/flashsac/appo/ppo 四个算法树的 +`g1_motion_tracking` 与 ppo `go2_joystick_flat` 共 10 个 leaf 违规,其中 +appo/ppo 的 `g1_motion_tracking` 目录连 `base.yaml` 都不存在,mujoco owner +被迫充当 base。 + +## Decision + +- 每个 `/task//` 目录必须包含一个 `base.yaml`,持有该任务对该算法的 + canonical contract:scene/observations/actions/commands/terminations/reward, + canonical DR `env.events`,以及后端共享的 training budget(`training.task_name`、 + `algo` 预算)。 +- 目录内每个仿真 backend leaf(`mujoco.yaml`、`mjwarp.yaml`、`genesis.yaml`、 + `motrix.yaml`、`newton.yaml`、`superdex.yaml`、`drake.yaml` 等)的 Hydra + `defaults:` 只允许列出 `/task//base` 与 `_self_`,禁止继承任何其他 + backend leaf。leaf 直接持有身份字段(`training.task_name`、 + `training.sim_backend`,见 ADR-0003)、渲染/设备 placement,以及显式支持 + 或禁用的 DR term(如 `events.push_robot: null`)。 +- 若某后端确有独立的调参需求(如 superdex 的 `max_iterations`、观测归一化), + 必须在该 leaf 内显式写出完整数值,不允许通过继承其他后端 owner 获得。 +- `base.yaml` 自身不是可训练入口(不持有 `training.sim_backend`),仅作为 + contract 声明与 leaf 的 compose 基线。 + +## Stable Contracts + +- `src/unilab/conf//task//.yaml` 的 `defaults:` 恰好包含 + `/task//base` 与 `_self_`;该结构由 + `tests/config/test_config_system.py` 的继承结构测试全量守护。 +- backend leaf 的组合结果必须能从 `base.yaml + leaf 文件本身` 完整推出; + review 时 leaf 文件即其完整 backend 契约。 +- 后端间共享的 DR/奖励定义收敛在 `base.yaml`;单个后端不支持某 term 时, + 在该后端 leaf 中显式置 `null` 或收窄参数,禁止"借道"其他后端 owner。 +- 跨算法对齐(如 flashsac 与 sac 的 `g1_motion_tracking` 任务契约)通过使两边 + `base.yaml` 一致来表达,而不是通过跨树继承。 + +## Alternatives Considered + +- **保留跨后端继承、仅加注释约束:** 拒绝。继承链使行为变更不可见,正是本次 + 清理要消除的问题;注释无法被测试守护。 +- **允许继承同任务任意 leaf、只禁止跨任务继承:** 拒绝。同任务跨后端继承同样 + 让 MuJoCo owner 的修改静默传播到其他后端,违背"leaf 文件即完整契约"目标。 +- **用 Hydra config group(`task/g1_motion_tracking` 组默认值)代替显式 + defaults:** 拒绝。组默认值是隐式路由,review 时同样看不到 leaf 的 compose + 基线;显式 `defaults: [/task//base, _self_]` 一目了然。 + +## Consequences + +- 后端 owner 之间出现少量显式数值重复(如 superdex 携带观测归一化与学习率)。 + 这是有意的:重复可见优于继承隐式。 +- 修改 `base.yaml` 等同于修改该任务所有后端的契约,必须按 ADR-0003 的 + owner 变更流程评审;修改单个 leaf 不再影响其他后端。 +- 新增 backend leaf 时必须先确认同目录 `base.yaml` 存在;目录内没有 + `base.yaml` 时禁止新增 leaf。 +- 2026-11-18 的迁移以"组合结果不变"为验收标准:除 + flashsac `g1_motion_tracking` 按本仓库决策向 sac 契约对齐(补齐 canonical DR、 + `geom_names`、actor 有偏关节观测)外,所有受影响 owner 的 Hydra 组合输出 + 与迁移前逐键一致。 + +## Evidence In Repo + +- owner 配置树:`src/unilab/conf/{sac,flashsac,appo,ppo}/task//`。 +- 对齐示例:flashsac `g1_motion_tracking` 的 `base.yaml` 与 sac 同任务 + `base.yaml` 契约一致(scene/observations/commands/reward/terminations/DR)。 +- 继承结构守护测试:`tests/config/test_config_system.py`。 +- 组合等价与任务契约测试:`tests/config/test_g1_motion_tracking_owners.py`。 + +## Related Documents + +- {doc}`ADR Index ` +- {doc}`ADR-0003 Task Owner And Config Compose Contract ` +- {doc}`RL Infrastructure 开发标准 ` diff --git a/docs/sphinx/source/en/2-user_guide/1-training/7-tensor_runtime_production.md b/docs/sphinx/source/en/2-user_guide/1-training/7-tensor_runtime_production.md index 739107ffb..244cc922a 100644 --- a/docs/sphinx/source/en/2-user_guide/1-training/7-tensor_runtime_production.md +++ b/docs/sphinx/source/en/2-user_guide/1-training/7-tensor_runtime_production.md @@ -47,9 +47,12 @@ the MJWarp owner inherits the MuJoCo FlashSAC owner and resolves to: | Motion | `motions/g1/dance1_subject2_part.npz` | The MuJoCo owner uses the same 2048-environment training budget, but publishes -collector metrics every step. The MJWarp task is also a DR-free lifecycle -baseline: its selected-row tensor reset does not negotiate reset randomization -or interval wrenches. +collector metrics every step. The MJWarp owner inherits the same canonical DR +(reset-mode model-field randomization plus the linear-velocity interval push) +and only zeroes the angular-velocity kicks, because its selected-row tensor +reset does not negotiate the interval angular root-velocity capability. The +Newton owner does not support reset randomization and sets the canonical DR +terms to `null` explicitly. At the current generated evidence grade, FlashSAC / `g1_motion_tracking` / MJWarp is `Configured`. A successful local soak does not diff --git a/docs/sphinx/source/en/2-user_guide/1-training/8-sync_free_autoreset.md b/docs/sphinx/source/en/2-user_guide/1-training/8-sync_free_autoreset.md index 14d2ea21b..5f6a29c86 100644 --- a/docs/sphinx/source/en/2-user_guide/1-training/8-sync_free_autoreset.md +++ b/docs/sphinx/source/en/2-user_guide/1-training/8-sync_free_autoreset.md @@ -179,6 +179,10 @@ env.reset_selection_blockers() # [] when the task can enable masked_autoreset ``` Then convert, replace or remove each named term. Reset-randomization events are -common blockers; the G1 motion-tracking FlashSAC owner -(`src/unilab/conf/flashsac/task/g1_motion_tracking/base.yaml`) is a DR-free -baseline for this reason. +common blockers; when a backend cannot negotiate them, the task owner must +disable those terms explicitly. The G1 motion-tracking owners keep the +canonical DR contract in +`src/unilab/conf//task/g1_motion_tracking/base.yaml`, and backends without +reset-randomization support (for example the FlashSAC Newton owner) set the +unsupported event terms to `null` in their own leaf instead of forking the +contract. diff --git a/docs/sphinx/source/zh_CN/2-user_guide/1-training/7-tensor_runtime_production.md b/docs/sphinx/source/zh_CN/2-user_guide/1-training/7-tensor_runtime_production.md index 97ee535dd..886efeccf 100644 --- a/docs/sphinx/source/zh_CN/2-user_guide/1-training/7-tensor_runtime_production.md +++ b/docs/sphinx/source/zh_CN/2-user_guide/1-training/7-tensor_runtime_production.md @@ -46,8 +46,10 @@ MJWarp owner 继承 FlashSAC MuJoCo owner,并解析为: | Motion | `motions/g1/dance1_subject2_part.npz` | MuJoCo owner 使用同样的 2048 环境训练预算,但每步发布 collector metric。 -MJWarp task 也是 DR-free lifecycle baseline:其 selected-row tensor reset -不协商 reset randomization 或 interval wrench。 +MJWarp owner 继承同样的 canonical DR(reset 模式模型字段随机化 + 线速度 +interval push),仅将角速度推扰收窄为零,因为其 selected-row tensor reset +不协商 interval angular root-velocity。Newton owner 不支持 reset 随机化, +将 canonical DR 显式置 `null`。 在当前生成证据等级中,FlashSAC / `g1_motion_tracking` / MJWarp 是 `Configured`。本地 soak 成功不会自动把它提升为 `Benchmarked` 或 diff --git a/src/unilab/conf/appo/task/g1_motion_tracking/base.yaml b/src/unilab/conf/appo/task/g1_motion_tracking/base.yaml new file mode 100644 index 000000000..293b198df --- /dev/null +++ b/src/unilab/conf/appo/task/g1_motion_tracking/base.yaml @@ -0,0 +1,236 @@ +# @package _global_ +# Canonical G1 WBT task declaration for APPO owners. Backend owners inherit +# this contract and override only backend identity; sim owners never inherit +# each other (ADR-0023). +training: + task_name: G1MotionTracking + play_steps: 1000 + +algo: + num_envs: 1024 + max_iterations: 5000 + save_interval: 500 + algorithm: + adaptive_kl_factor: 2.0 + adaptive_lr_factor: 1.5 + +env: + scene: + model_file: src/unilab/assets/robots/g1/scene_flat.xml + default_keyframe_name: stand + entities: + robot: + root_body_name: pelvis + joint_names: &g1_joints + - left_hip_pitch_joint + - left_hip_roll_joint + - left_hip_yaw_joint + - left_knee_joint + - left_ankle_pitch_joint + - left_ankle_roll_joint + - right_hip_pitch_joint + - right_hip_roll_joint + - right_hip_yaw_joint + - right_knee_joint + - right_ankle_pitch_joint + - right_ankle_roll_joint + - waist_yaw_joint + - waist_roll_joint + - waist_pitch_joint + - left_shoulder_pitch_joint + - left_shoulder_roll_joint + - left_shoulder_yaw_joint + - left_elbow_joint + - left_wrist_roll_joint + - left_wrist_pitch_joint + - left_wrist_yaw_joint + - right_shoulder_pitch_joint + - right_shoulder_roll_joint + - right_shoulder_yaw_joint + - right_elbow_joint + - right_wrist_roll_joint + - right_wrist_pitch_joint + - right_wrist_yaw_joint + actuator_names: *g1_joints + body_names: &tracked_bodies + - pelvis + - left_hip_roll_link + - left_knee_link + - left_ankle_roll_link + - right_hip_roll_link + - right_knee_link + - right_ankle_roll_link + - torso_link + - left_shoulder_roll_link + - left_elbow_link + - left_wrist_yaw_link + - right_shoulder_roll_link + - right_elbow_link + - right_wrist_yaw_link + geom_names: + - left_foot1_collision + - left_foot2_collision + - left_foot3_collision + - left_foot4_collision + - left_foot5_collision + - left_foot6_collision + - left_foot7_collision + - right_foot1_collision + - right_foot2_collision + - right_foot3_collision + - right_foot4_collision + - right_foot5_collision + - right_foot6_collision + - right_foot7_collision + sim_dt: 0.006666666666666667 + ctrl_dt: 0.02 + max_episode_seconds: 10.0 + observations: + actor: + terms: + command: &command_obs + func: unilab.envs.mdp.generated_commands + params: {command_name: motion} + motion_anchor_pos_b: &anchor_pos_obs + func: unilab.tasks.motion_tracking.common.manager_terms.motion_anchor_pos_b + params: {command_name: motion} + motion_anchor_ori_b: &anchor_ori_obs + func: unilab.tasks.motion_tracking.common.manager_terms.motion_anchor_ori_b + params: {command_name: motion} + base_lin_vel: &base_lin_vel_obs + func: unilab.envs.mdp.builtin_sensor + params: {sensor_name: pelvis_local_linvel, entity_name: robot} + base_ang_vel: &base_ang_vel_obs + func: unilab.envs.mdp.builtin_sensor + params: {sensor_name: torso_gyro, entity_name: robot} + joint_pos: &joint_pos_obs + func: unilab.tasks.motion_tracking.common.manager_terms.motion_joint_pos_rel + params: {command_name: motion} + joint_vel: &joint_vel_obs + func: unilab.envs.mdp.joint_vel_rel + actions: &actions_obs + func: unilab.envs.mdp.last_action + critic: + terms: + command: *command_obs + motion_anchor_pos_b: *anchor_pos_obs + motion_anchor_ori_b: *anchor_ori_obs + base_lin_vel: *base_lin_vel_obs + base_ang_vel: *base_ang_vel_obs + joint_pos: *joint_pos_obs + joint_vel: *joint_vel_obs + actions: *actions_obs + body_pos: + func: unilab.tasks.motion_tracking.common.manager_terms.robot_body_pos_b + params: {command_name: motion} + body_ori: + func: unilab.tasks.motion_tracking.common.manager_terms.robot_body_ori_b + params: {command_name: motion} + actions: + joint_pos: + _target_: unilab.tasks.motion_tracking.common.manager_terms.MotionJointPositionActionCfg + entity_name: robot + actuator_names: [".*"] + scale: 0.25 + use_default_offset: true + command_name: motion + commands: + motion: + _target_: unilab.tasks.motion_tracking.common.manager_terms.MotionCommandCfg + entity_name: robot + resampling_time_range: [1.0e9, 1.0e9] + params: + motion_file: motions/g1/dance1_subject2_part.npz + anchor_body_name: torso_link + body_names: *tracked_bodies + sampling_mode: adaptive + sampling_start_ratio: 0.0 + truncate_on_clip_end: false + pose_range: + x: [-0.05, 0.05] + y: [-0.05, 0.05] + z: [-0.01, 0.01] + roll: [-0.1, 0.1] + pitch: [-0.1, 0.1] + yaw: [-0.2, 0.2] + velocity_range: + x: [-0.5, 0.5] + y: [-0.5, 0.5] + z: [-0.2, 0.2] + roll: [-0.52, 0.52] + pitch: [-0.52, 0.52] + yaw: [-0.78, 0.78] + joint_position_range: [-0.1, 0.1] + joint_default_position_range: [0.0, 0.0] + terminations: + time_out: + func: unilab.envs.mdp.time_out + time_out: true + anchor_pos: + func: unilab.tasks.motion_tracking.common.manager_terms.BadAnchorPosZOnly + params: {command_name: motion, threshold: 0.25} + anchor_ori: + func: unilab.tasks.motion_tracking.common.manager_terms.bad_anchor_ori + params: + command_name: motion + threshold: 0.8 + asset_cfg: + _target_: unilab.managers.SceneEntityCfg + name: robot + ee_body_pos: + func: unilab.tasks.motion_tracking.common.manager_terms.BadMotionBodyPosZOnly + params: + command_name: motion + threshold: 0.25 + body_names: + - left_ankle_roll_link + - right_ankle_roll_link + - left_wrist_yaw_link + - right_wrist_yaw_link + policy_observation_group: actor + critic_observation_group: critic + +reward: + motion_global_root_pos: + func: unilab.tasks.motion_tracking.common.manager_terms.motion_global_anchor_position_error_exp + weight: 0.5 + params: {command_name: motion, std: 0.3} + motion_global_root_ori: + func: unilab.tasks.motion_tracking.common.manager_terms.motion_global_anchor_orientation_error_exp + weight: 0.5 + params: {command_name: motion, std: 0.4} + motion_body_pos: + func: unilab.tasks.motion_tracking.common.manager_terms.MotionRelativeBodyPositionErrorExp + weight: 1.0 + params: {command_name: motion, std: 0.3} + motion_body_ori: + func: unilab.tasks.motion_tracking.common.manager_terms.MotionRelativeBodyOrientationErrorExp + weight: 1.0 + params: {command_name: motion, std: 0.4} + motion_body_lin_vel: + func: unilab.tasks.motion_tracking.common.manager_terms.MotionGlobalBodyLinearVelocityErrorExp + weight: 1.0 + params: {command_name: motion, std: 1.0} + motion_body_ang_vel: + func: unilab.tasks.motion_tracking.common.manager_terms.MotionGlobalBodyAngularVelocityErrorExp + weight: 1.0 + params: {command_name: motion, std: 3.14} + motion_joint_pos: + func: unilab.tasks.motion_tracking.common.manager_terms.motion_joint_position_error_exp + weight: 0.0 + params: {command_name: motion, std: 0.2} + motion_joint_vel: + func: unilab.tasks.motion_tracking.common.manager_terms.motion_joint_velocity_error_exp + weight: 0.0 + params: {command_name: motion, std: 1.0} + action_rate_l2: + func: unilab.envs.mdp.action_rate_l2 + weight: -0.1 + joint_limit: + func: unilab.tasks.motion_tracking.common.manager_terms.joint_pos_limits + weight: -10.0 + params: + asset_cfg: + _target_: unilab.managers.SceneEntityCfg + name: robot + joint_names: ".*" diff --git a/src/unilab/conf/appo/task/g1_motion_tracking/motrix.yaml b/src/unilab/conf/appo/task/g1_motion_tracking/motrix.yaml index 22bbdadfa..01043ec56 100644 --- a/src/unilab/conf/appo/task/g1_motion_tracking/motrix.yaml +++ b/src/unilab/conf/appo/task/g1_motion_tracking/motrix.yaml @@ -1,6 +1,6 @@ # @package _global_ defaults: - - /task/g1_motion_tracking/mujoco + - /task/g1_motion_tracking/base - _self_ training: diff --git a/src/unilab/conf/appo/task/g1_motion_tracking/mujoco.yaml b/src/unilab/conf/appo/task/g1_motion_tracking/mujoco.yaml index ea0bd85fe..d7a3ad298 100644 --- a/src/unilab/conf/appo/task/g1_motion_tracking/mujoco.yaml +++ b/src/unilab/conf/appo/task/g1_motion_tracking/mujoco.yaml @@ -1,234 +1,9 @@ # @package _global_ +# MuJoCo owner for G1 WBT with APPO; the full task contract lives in base.yaml. +defaults: + - /task/g1_motion_tracking/base + - _self_ + training: task_name: G1MotionTracking sim_backend: mujoco - play_steps: 1000 - -algo: - num_envs: 1024 - max_iterations: 5000 - save_interval: 500 - algorithm: - adaptive_kl_factor: 2.0 - adaptive_lr_factor: 1.5 - -env: - scene: - model_file: src/unilab/assets/robots/g1/scene_flat.xml - default_keyframe_name: stand - entities: - robot: - root_body_name: pelvis - joint_names: &g1_joints - - left_hip_pitch_joint - - left_hip_roll_joint - - left_hip_yaw_joint - - left_knee_joint - - left_ankle_pitch_joint - - left_ankle_roll_joint - - right_hip_pitch_joint - - right_hip_roll_joint - - right_hip_yaw_joint - - right_knee_joint - - right_ankle_pitch_joint - - right_ankle_roll_joint - - waist_yaw_joint - - waist_roll_joint - - waist_pitch_joint - - left_shoulder_pitch_joint - - left_shoulder_roll_joint - - left_shoulder_yaw_joint - - left_elbow_joint - - left_wrist_roll_joint - - left_wrist_pitch_joint - - left_wrist_yaw_joint - - right_shoulder_pitch_joint - - right_shoulder_roll_joint - - right_shoulder_yaw_joint - - right_elbow_joint - - right_wrist_roll_joint - - right_wrist_pitch_joint - - right_wrist_yaw_joint - actuator_names: *g1_joints - body_names: &tracked_bodies - - pelvis - - left_hip_roll_link - - left_knee_link - - left_ankle_roll_link - - right_hip_roll_link - - right_knee_link - - right_ankle_roll_link - - torso_link - - left_shoulder_roll_link - - left_elbow_link - - left_wrist_yaw_link - - right_shoulder_roll_link - - right_elbow_link - - right_wrist_yaw_link - geom_names: - - left_foot1_collision - - left_foot2_collision - - left_foot3_collision - - left_foot4_collision - - left_foot5_collision - - left_foot6_collision - - left_foot7_collision - - right_foot1_collision - - right_foot2_collision - - right_foot3_collision - - right_foot4_collision - - right_foot5_collision - - right_foot6_collision - - right_foot7_collision - sim_dt: 0.006666666666666667 - ctrl_dt: 0.02 - max_episode_seconds: 10.0 - observations: - actor: - terms: - command: &command_obs - func: unilab.envs.mdp.generated_commands - params: {command_name: motion} - motion_anchor_pos_b: &anchor_pos_obs - func: unilab.tasks.motion_tracking.common.manager_terms.motion_anchor_pos_b - params: {command_name: motion} - motion_anchor_ori_b: &anchor_ori_obs - func: unilab.tasks.motion_tracking.common.manager_terms.motion_anchor_ori_b - params: {command_name: motion} - base_lin_vel: &base_lin_vel_obs - func: unilab.envs.mdp.builtin_sensor - params: {sensor_name: pelvis_local_linvel, entity_name: robot} - base_ang_vel: &base_ang_vel_obs - func: unilab.envs.mdp.builtin_sensor - params: {sensor_name: torso_gyro, entity_name: robot} - joint_pos: &joint_pos_obs - func: unilab.tasks.motion_tracking.common.manager_terms.motion_joint_pos_rel - params: {command_name: motion} - joint_vel: &joint_vel_obs - func: unilab.envs.mdp.joint_vel_rel - actions: &actions_obs - func: unilab.envs.mdp.last_action - critic: - terms: - command: *command_obs - motion_anchor_pos_b: *anchor_pos_obs - motion_anchor_ori_b: *anchor_ori_obs - base_lin_vel: *base_lin_vel_obs - base_ang_vel: *base_ang_vel_obs - joint_pos: *joint_pos_obs - joint_vel: *joint_vel_obs - actions: *actions_obs - body_pos: - func: unilab.tasks.motion_tracking.common.manager_terms.robot_body_pos_b - params: {command_name: motion} - body_ori: - func: unilab.tasks.motion_tracking.common.manager_terms.robot_body_ori_b - params: {command_name: motion} - actions: - joint_pos: - _target_: unilab.tasks.motion_tracking.common.manager_terms.MotionJointPositionActionCfg - entity_name: robot - actuator_names: [".*"] - scale: 0.25 - use_default_offset: true - command_name: motion - commands: - motion: - _target_: unilab.tasks.motion_tracking.common.manager_terms.MotionCommandCfg - entity_name: robot - resampling_time_range: [1.0e9, 1.0e9] - params: - motion_file: motions/g1/dance1_subject2_part.npz - anchor_body_name: torso_link - body_names: *tracked_bodies - sampling_mode: adaptive - sampling_start_ratio: 0.0 - truncate_on_clip_end: false - pose_range: - x: [-0.05, 0.05] - y: [-0.05, 0.05] - z: [-0.01, 0.01] - roll: [-0.1, 0.1] - pitch: [-0.1, 0.1] - yaw: [-0.2, 0.2] - velocity_range: - x: [-0.5, 0.5] - y: [-0.5, 0.5] - z: [-0.2, 0.2] - roll: [-0.52, 0.52] - pitch: [-0.52, 0.52] - yaw: [-0.78, 0.78] - joint_position_range: [-0.1, 0.1] - joint_default_position_range: [0.0, 0.0] - terminations: - time_out: - func: unilab.envs.mdp.time_out - time_out: true - anchor_pos: - func: unilab.tasks.motion_tracking.common.manager_terms.BadAnchorPosZOnly - params: {command_name: motion, threshold: 0.25} - anchor_ori: - func: unilab.tasks.motion_tracking.common.manager_terms.bad_anchor_ori - params: - command_name: motion - threshold: 0.8 - asset_cfg: - _target_: unilab.managers.SceneEntityCfg - name: robot - ee_body_pos: - func: unilab.tasks.motion_tracking.common.manager_terms.BadMotionBodyPosZOnly - params: - command_name: motion - threshold: 0.25 - body_names: - - left_ankle_roll_link - - right_ankle_roll_link - - left_wrist_yaw_link - - right_wrist_yaw_link - policy_observation_group: actor - critic_observation_group: critic - -reward: - motion_global_root_pos: - func: unilab.tasks.motion_tracking.common.manager_terms.motion_global_anchor_position_error_exp - weight: 0.5 - params: {command_name: motion, std: 0.3} - motion_global_root_ori: - func: unilab.tasks.motion_tracking.common.manager_terms.motion_global_anchor_orientation_error_exp - weight: 0.5 - params: {command_name: motion, std: 0.4} - motion_body_pos: - func: unilab.tasks.motion_tracking.common.manager_terms.MotionRelativeBodyPositionErrorExp - weight: 1.0 - params: {command_name: motion, std: 0.3} - motion_body_ori: - func: unilab.tasks.motion_tracking.common.manager_terms.MotionRelativeBodyOrientationErrorExp - weight: 1.0 - params: {command_name: motion, std: 0.4} - motion_body_lin_vel: - func: unilab.tasks.motion_tracking.common.manager_terms.MotionGlobalBodyLinearVelocityErrorExp - weight: 1.0 - params: {command_name: motion, std: 1.0} - motion_body_ang_vel: - func: unilab.tasks.motion_tracking.common.manager_terms.MotionGlobalBodyAngularVelocityErrorExp - weight: 1.0 - params: {command_name: motion, std: 3.14} - motion_joint_pos: - func: unilab.tasks.motion_tracking.common.manager_terms.motion_joint_position_error_exp - weight: 0.0 - params: {command_name: motion, std: 0.2} - motion_joint_vel: - func: unilab.tasks.motion_tracking.common.manager_terms.motion_joint_velocity_error_exp - weight: 0.0 - params: {command_name: motion, std: 1.0} - action_rate_l2: - func: unilab.envs.mdp.action_rate_l2 - weight: -0.1 - joint_limit: - func: unilab.tasks.motion_tracking.common.manager_terms.joint_pos_limits - weight: -10.0 - params: - asset_cfg: - _target_: unilab.managers.SceneEntityCfg - name: robot - joint_names: ".*" diff --git a/src/unilab/conf/flashsac/task/g1_motion_tracking/base.yaml b/src/unilab/conf/flashsac/task/g1_motion_tracking/base.yaml index a791b3d14..7fc2b4da5 100644 --- a/src/unilab/conf/flashsac/task/g1_motion_tracking/base.yaml +++ b/src/unilab/conf/flashsac/task/g1_motion_tracking/base.yaml @@ -1,10 +1,26 @@ # @package _global_ -# Canonical G1 Whole-Body Tracking (WBT) Manager-Based task declaration for -# FlashSAC owners. It intentionally shares the SAC scene, command, reward, and -# termination contract, but uses unbiased actor/critic joint observations and -# a DR-free lifecycle baseline: the current mjwarp tensor mask reset -# does not negotiate reset randomization or interval wrenches. Backend owner -# leaves inherit this file and only override backend/algo tuning. +# Canonical G1 Whole-Body Tracking (WBT) task declaration for FlashSAC owners. +# Aligned with the SAC owner contract (scene, observations, commands, rewards, +# terminations, and canonical training DR) so run metrics are directly +# comparable across algos; the FlashSAC training budget matches the SAC owner +# (2048 envs x 25000 iterations, gamma 0.99) while FlashSAC algo-specific +# knobs (batch_size, num_atoms, temp/n_step params) stay at the +# conf/flashsac/config.yaml defaults. Backend owners inherit this contract and +# override only backend identity, rendering, or explicitly supported/disabled +# DR terms; sim owners never inherit each other (ADR-0023). + +training: + task_name: G1MotionTracking + +algo: + num_envs: 2048 + max_iterations: 25000 + save_interval: 1000 + gamma: 0.99 + tau: 0.05 + updates_per_step: 4 + policy_frequency: 2 + env: seed: null scene: @@ -13,6 +29,22 @@ env: entities: robot: root_body_name: pelvis + # Foot geom metadata is required by backend friction DR bindings. + geom_names: + - left_foot1_collision + - left_foot2_collision + - left_foot3_collision + - left_foot4_collision + - left_foot5_collision + - left_foot6_collision + - left_foot7_collision + - right_foot1_collision + - right_foot2_collision + - right_foot3_collision + - right_foot4_collision + - right_foot5_collision + - right_foot6_collision + - right_foot7_collision joint_names: &g1_joints - left_hip_pitch_joint - left_hip_roll_joint @@ -89,8 +121,8 @@ env: _target_: unilab.managers._noise.UniformNoiseCfg n_min: -0.2 n_max: 0.2 - joint_pos: &joint_pos_obs - func: unilab.tasks.motion_tracking.common.manager_terms.motion_joint_pos_rel + joint_pos: + func: unilab.tasks.motion_tracking.common.manager_terms.motion_joint_pos_rel_biased params: {command_name: motion} noise: _target_: unilab.managers._noise.UniformNoiseCfg @@ -111,9 +143,11 @@ env: motion_anchor_ori_b: *anchor_ori_obs base_lin_vel: *base_lin_vel_obs base_ang_vel: *base_ang_vel_obs - joint_pos: *joint_pos_obs joint_vel: *joint_vel_obs actions: *actions_obs + joint_pos: + func: unilab.tasks.motion_tracking.common.manager_terms.motion_joint_pos_rel + params: {command_name: motion} body_pos: func: unilab.tasks.motion_tracking.common.manager_terms.robot_body_pos_b params: {command_name: motion} @@ -188,6 +222,55 @@ env: - right_ankle_roll_link - left_wrist_yaw_link - right_wrist_yaw_link + # mjlab fixes model-field biases at startup. UniLab model-field writes are + # reset-transaction capabilities, so those terms use reset mode. Backend + # owners override only the terms their backend cannot negotiate. + events: + base_com: + func: unilab.envs.mdp.randomize_rigid_body_com + mode: reset + params: + asset_cfg: + _target_: unilab.managers.SceneEntityCfg + name: robot + body_names: torso_link + com_range: + x: [-0.025, 0.025] + y: [-0.05, 0.05] + z: [-0.05, 0.05] + encoder_bias: + func: unilab.envs.mdp.randomize_encoder_bias + mode: reset + params: + bias_range: [-0.01, 0.01] + asset_cfg: + _target_: unilab.managers.SceneEntityCfg + name: robot + joint_names: ".*" + foot_friction: + func: unilab.envs.mdp.geom_friction + mode: reset + params: + asset_cfg: + _target_: unilab.managers.SceneEntityCfg + name: robot + geom_names: "^(left|right)_foot[1-7]_collision$" + ranges: [0.3, 1.2] + operation: abs + shared_random: true + push_robot: + func: unilab.envs.mdp.push_by_setting_velocity + mode: interval + interval_range_s: [1.0, 3.0] + is_global_time: true + params: + velocity_range: + x: [-0.5, 0.5] + y: [-0.5, 0.5] + z: [-0.2, 0.2] + roll: [-0.52, 0.52] + pitch: [-0.52, 0.52] + yaw: [-0.78, 0.78] policy_observation_group: actor critic_observation_group: critic diff --git a/src/unilab/conf/flashsac/task/g1_motion_tracking/genesis.yaml b/src/unilab/conf/flashsac/task/g1_motion_tracking/genesis.yaml index d208e3066..8ef663e31 100644 --- a/src/unilab/conf/flashsac/task/g1_motion_tracking/genesis.yaml +++ b/src/unilab/conf/flashsac/task/g1_motion_tracking/genesis.yaml @@ -5,11 +5,16 @@ # §3.3), so the integrator g1.xml declares is re-declared here: the timestep # flows through the existing base.yaml sim_dt -> ctrl_dt chain, and the # integrator through the env field below. Requires unisim-core>=1.5.1 for the -# motion body id capability (unilabsim/unisim#137). Playback uses Genesis +# motion body id capability (unilabsim/unisim#137). Mirrors the SAC Genesis +# owner DR subset: startup pelvis/torso mass scaling, inherited torso COM +# reset, reset PD-gain scaling, and the six-axis interval velocity kick. +# Absolute geom friction remains intentionally fail-closed because Genesis has +# no equivalent per-environment friction API; encoder_bias is a host-side +# observation bias and stays enabled. Playback uses Genesis # native rendering: with a display, play_render_mode=auto opens the # interactive viewer; headless hosts fall back to offscreen camera recording. defaults: - - /task/g1_motion_tracking/mujoco + - /task/g1_motion_tracking/base - _self_ training: @@ -18,6 +23,8 @@ training: play_render_mode: auto # First tick includes collector env construction; genesis JIT startup exceeds # the 30s default (same override as sac g1_walk_flat/genesis). + +env: # GPU device id for the process-wide Genesis session. Multi-GPU training # overrides this cold-path field from the rank topology; zero preserves the # single-device owner default. @@ -25,3 +32,29 @@ training: # Re-declares the MJCF