Skip to content

Fix Agibot left-arm teleop defects and env_cfg_callback contract - #1110

Open
qianl-nv wants to merge 4 commits into
mainfrom
chuanruiz/fix/agibot_left_arm_and_env_cfg_callback
Open

Fix Agibot left-arm teleop defects and env_cfg_callback contract#1110
qianl-nv wants to merge 4 commits into
mainfrom
chuanruiz/fix/agibot_left_arm_and_env_cfg_callback

Conversation

@qianl-nv

@qianl-nv qianl-nv commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Summary

Fix three defects that make Agibot left-arm teleoperation unusable and crash documented env_cfg_callback usage.

Detailed description

  • Add a robot-only reset event to AgibotEmbodiment: event_config was never set, so every joint reset to 0 and RMPFlow's lula model no longer described the actual robot. Robot-only rather than mdp.reset_scene_to_default, which would also reset every scene asset and trample task-level contracts such as disable_reset_pose.
  • Fix the left arm's frame/body offset quaternions, written as (w,x,y,z) in (x,y,z,w) fields: read as a 180° flip, the arm ran away ~270 mm under a zero command after every reset (corrected: 9.6 mm) and could not be teleoperated.
  • ArenaEnvBuilder accept an env_cfg_callback that must return a cfg. Update the doc and comments to state so.
  • Harden test_place_upright_mug_multi to assert the env_ids/percentage plumbing at the moment of the write: it previously depended on marginal rotate-in-place physics that any change to the robot's reset behaviour flips.

Resubmitted for #1097 since the CI doesn't work from fork.

Add a robot-only reset event to AgibotEmbodiment (event_config was never
set, so every joint reset to 0 and RMPFlow's lula model no longer
described the actual robot). Fix the left arm's frame/body offset
quaternions, previously written as (w,x,y,z) in (x,y,z,w) fields and
read as a 180 deg flip. Let ArenaEnvBuilder accept an env_cfg_callback
that mutates the config in place and returns None, as the documented
example does. Harden test_place_upright_mug_multi to assert the
env_ids/percentage plumbing at the moment of the write instead of after
marginal rotate-in-place physics.

Signed-off-by: Chuanrui Zhang <chuanruiz@nvidia.com>
…back return

Per review: the xyzw fix needs no inline explanation (the wxyz values were
an oversight from the Lab 3.0 order switch), and env_cfg_callback now has
one contract -- it must return the config -- instead of the builder
tolerating a None return. Doc example and type hint updated to match;
test docstring shortened.

Signed-off-by: Chuanrui Zhang <chuanruiz@nvidia.com>
@greptile-apps

greptile-apps Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR repairs Agibot left-arm reset and frame-offset behavior while clarifying the environment-config callback contract.

  • Adds an Agibot-specific reset event that restores default robot joint state and targets without resetting unrelated scene assets.
  • Corrects the left-arm frame-transformer and RMPFlow body-offset quaternions.
  • Documents and enforces that env_cfg_callback returns an environment configuration.
  • Revises the multi-environment upright-placement test to validate selection and percentage plumbing immediately after writes.

Confidence Score: 5/5

The PR appears safe to merge with no concrete blocking or non-blocking defects identified.

The reset configuration, paired left-arm offsets, callback contract updates, and targeted test changes are internally consistent, and no reachable changed-code failure was established.

Important Files Changed

Filename Overview
isaaclab_arena/embodiments/agibot/agibot.py Adds robot-only reset behavior and consistently corrects the left-arm controller and observation-frame offsets; no actionable defect was established.
isaaclab_arena/environments/arena_env_builder.py Adds a fail-fast assertion for callbacks that violate the existing return-value contract.
isaaclab_arena/environments/isaaclab_arena_environment.py Updates callback typing and documentation to accurately express the builder's return-value contract.
isaaclab_arena/tests/test_place_upright_task.py Refocuses the multi-environment test on immediate write plumbing rather than subsequent physics evolution.
docs/pages/concepts/environment/environment_definition.rst Corrects the callback example by explicitly returning the mutated configuration.

Reviews (1): Last reviewed commit: "Simplify agibot reset comments" | Re-trigger Greptile

)


def reset_robot_to_default(env, env_ids, asset_cfg: SceneEntityCfg = SceneEntityCfg("robot")):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 A third, divergent "reset robot to default" implementation

We now have three approaches to the same RMPFlow "reset joints to their default" need: Franka and Galbot reuse franka_stack_events.randomize_joint_by_gaussian_offset (std=0), terms/events.py has reset_all_articulation_joints, and this adds a fourth bespoke one. The two things that make this one different are worth pulling out: it's robot-only (so it doesn't trample task assets) and it also restores the joint position/velocity targets. That last part is the interesting bit — if RMPFlow needs the targets reset too, then Galbot's gaussian-offset reset (which doesn't touch targets) likely has the same latent runaway you're fixing here.

Could this live in the shared terms/events.py as a robot-only reset_joints_to_default (with the target restore) so Galbot/Franka and any future RMPFlow embodiment converge on one implementation instead of drifting? Happy to be wrong if the target reset is genuinely Agibot-specific.

)
"""Restore ``init_state`` joint values and targets on every reset.

RMPFlow expects `joint_lift_body`` and ``joint_body_pitch`` at those default values

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Mismatched backticks

`joint_lift_body `` opens with one backtick and closes with two, so it renders wrong. Also worth escaping the leading underscore names if this is meant as RST.

Suggested change
RMPFlow expects `joint_lift_body`` and ``joint_body_pitch`` at those default values
RMPFlow expects ``joint_lift_body`` and ``joint_body_pitch`` at those default values

@arena-review-bot

Copy link
Copy Markdown
Contributor

🤖 Isaac Lab-Arena Review Bot

Summary

This PR fixes three real Agibot left-arm teleop defects (a never-set event_config so RMPFlow no longer saw the true joint state, and two left-arm offset quaternions written in the wrong component order) and tightens the env_cfg_callback contract to require returning the config. The quaternion fix is backed by the author's own runaway-distance measurements (270 mm → 9.6 mm) and the callback change is documented and asserted — both look right. Overall a clean, well-scoped fix; the one thing worth a look is that the new reset helper is a third divergent take on a problem two other embodiments already solve differently.

Findings

🟡 Warning: embodiments/agibot/agibot.py:151reset_robot_to_default is a fourth bespoke "reset joints to default" implementation alongside Franka/Galbot's randomize_joint_by_gaussian_offset(std=0) and terms/events.py:reset_all_articulation_joints. The distinguishing behavior — robot-only + restoring joint targets — is exactly what the others lack, so Galbot's target-less reset may carry the same latent runaway. Consider hoisting a robot-only reset_joints_to_default into the shared terms/events.py so RMPFlow embodiments converge on one path.

🔵 Improvement: embodiments/agibot/agibot.py:172 — mismatched backticks in the attribute docstring ( `joint_lift_body `` opens with one, closes with two).

Test Coverage

test_place_upright_mug_multi is hardened to assert the env_ids/upright_percentage plumbing at write time instead of after marginal rotate-in-place physics — a good call. NUM_STEPS and step_zeros_and_call are still used by the other two tests, so no dead code is left behind. No test exercises the new Agibot reset path directly, but that path needs a live sim + RMPFlow and is hard to unit-test; the empirical teleop verification is a reasonable substitute for an alpha fix.

Verdict

Ship it (consider the shared-reset consolidation as a follow-up).

qianl-nv and others added 2 commits August 20, 2026 11:15
Per review: the term is neither Agibot- nor robot-specific, so it moves
next to the other event terms as
reset_joint_position_and_velocity_to_defaults. Also fixes a backtick
typo in the AgibotEventCfg docstring.

Signed-off-by: Chuanrui Zhang <chuanruiz@nvidia.com>
Comment thread isaaclab_arena/embodiments/agibot/agibot.py Outdated
Comment thread isaaclab_arena/environments/isaaclab_arena_environment.py
@qianl-nv
qianl-nv force-pushed the chuanruiz/fix/agibot_left_arm_and_env_cfg_callback branch from 202e197 to 81b96ad Compare August 21, 2026 10:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants