[kevinzakka]

Click a body to select it, Cmd/Ctrl+right-drag to move it, and Cmd/Ctrl+left-drag to rotate it. Hold Shift for the horizontal plane. Everything except the input goes through MuJoCo's own mjv_* perturbation functions, so it behaves like simulate. MuJoCo's translation connector is hidden for now because viser always draws its own drag arrow.

[Flova]

I am very much looking forward to this. Are there plans to integrate an xy plane drag that allows the robot to be repositioned without changing its height or rotation? Other simulators features this and I think it is also missing from the official mujoco viewer.

[kevinzakka]

@Flova it actually does exist in the native viewer but it's modifier-gated. Ctrl+(right-drag) translates in the vertical plane (mjMOUSE_MOVE_V) and Ctrl+Shift+(right-drag) is a horizontal-plane move (mjMOUSE_MOVE_H).

The blocker is on the viser side, not MuJoCo: viser only projects the cursor onto a camera-aligned plane and freezes the drag modifier at drag-start, so we can't reproduce the "Shift switches to the world-horizontal plane mid-drag" behavior. We'd need viser to expose the drag-plane normal (or allow live modifiers during a drag).

cc @brentyi

[brentyi]

Live modifiers: doable!
Drag-plane normal: this is also possible to support natively in Viser, but seems [initially] implementable without? In the mjviser callback, we can take the camera-aligned drag vector and add some modifier-conditioned projection logic.

[Flova]

@Flova it actually does exist in the native viewer but it's modifier-gated. Ctrl+(right-drag) translates in the vertical plane (mjMOUSE_MOVE_V) and Ctrl+Shift+(right-drag) is a horizontal-plane move (mjMOUSE_MOVE_H).

Maybe I am missing something, but I tried this in the native viewer and while the force vector stays in the plane, it only applies a sidewards force, which is not that great if you e.g. want to reposition a humanoid in a (maybe also stopped) simulator without messing with its stability. Webots for example supports both force/torque based interaction and "absolute" changing of the environment relative to the ground plane. It would be really great to have a feature like this for higher level robot behavior tests with mujoco.