Skip to content

Add interactive body dragging with MjvPerturb forces and mocap support - #33

Closed
shawn-maybush wants to merge 3 commits into
mujocolab:mainfrom
colony-core:feat/body-drag
Closed

shawn-maybush wants to merge 3 commits into
mujocolab:mainfrom
colony-core:feat/body-drag

Conversation

@shawn-maybush

Copy link
Copy Markdown

Summary

Adds interactive 3D body clicking and dragging using Viser's SceneNodeHandle.on_drag API (viser>=1.0.30), respecting the two-layer architecture between ViserMujocoScene and Viewer:

  • ViserMujocoScene (src/mjviser/scene.py):
    • Introduces BodyDragEvent (exported from mjviser) and add_body_drag_callback / on_body_drag / remove_body_drag_callback / remove_body_drag_callbacks.
    • Binds drag handlers across batched dynamic mesh groups (_mesh_groups), fixed body geom handles (_fixed_geom_handles), and dynamic convex-hull handles (_hull_dynamic_handles), automatically re-binding when visual or hull handles are rebuilt.
    • Resolves batched instance_index values back to (env_idx, body_id, mocap_id) for both full-batch and show_only_selected slicing modes, and subtracts scene_offset so callback coordinates are reported in MuJoCo world space.
  • Viewer (src/mjviser/viewer.py):
    • Registers default cmd/ctrl (pull & tumble) and cmd/ctrl+shift (rotate) drag callbacks backed by MuJoCo's native MjvPerturb, mjv_applyPerturbPose, and mjv_applyPerturbForce for mass-scaled, velocity-damped spring perturbations and direct mocap / paused free-body manipulation.
    • Normalizes MjvPerturb.localmass by model.vis.map.stiffness so perturbation_stiffness remains in $\text{N}/(\text{m}\cdot\text{kg})$ even when a model defines a custom <map stiffness=.../>.
    • Renders a 3D perturbation indicator line (/perturbation_force) while dragging dynamic bodies and adds a Perturbation folder under the Physics tab with Stiffness and Torque Scale sliders.
    • Clears active drag state and applied forces on simulation reset or keyframe load.

Test Plan

  • uv run ruff format --check & uv run ruff check
  • uv run pyright (0 errors)
  • uv run pytest (83 passed, including new unit tests in tests/test_scene.py and tests/test_viewer_drag.py covering multi-env instance decoding, convex-hull handle dragging, selective callback removal, spring/torque perturbations, damping, custom <map stiffness=.../>, mocap manipulation, and paused single-stepping)

… Viser private internals

Normalize MjvPerturb.localmass by model.vis.map.stiffness so perturbation_stiffness stays in N/(m*kg) across models with custom map stiffness, apply active drag forces when single-stepping while paused, hide the perturbation force line when paused, and track live drag handle IDs in ViserMujocoScene instead of reading handle._impl.removed.
@shawn-maybush

Copy link
Copy Markdown
Author

I closed as PR #21 is more comprehensive and further along.

@shawn-maybush
shawn-maybush deleted the feat/body-drag branch October 18, 2026 05:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant