feat: add viser as new manipulation vis backend#2475
Conversation
# Conflicts: # dimos/manipulation/planning/monitor/world_monitor.py
TomCC7
force-pushed
the
cc/feat/viser-vis-rework
branch
from
283a6da to
9087b4d
Compare
TomCC7
changed the title
These cosmetic-theme failures were buried in logger.debug (hidden by default). Raise them to warning and narrow the broad excepts to the expected theme-API-unavailable exceptions.
evaluate_joint_target set both "pose" and "ee_pose" to the same value (also calling get_ee_pose twice). Keep the single "ee_pose" key used by the error returns and read it directly in the GUI.
self.scene is a ViserManipulationScene; has_reference_grid, ensure_target_controls and set_target_visual_state always exist on it, so the hasattr guards were noise. The existing 'scene is None' checks suffice. Tests' minimal scene stand-ins gain the now-required methods.
…cept RobotModelConfig defines name, end_effector_link, base_link, home_joints and joint_limits_lower/upper, so the try/except AttributeError probes in the adapter and panel GUI never fired -- they were noise that defeated mypy. Access the fields directly. Tests gain a make_robot_config() builder so their config stand-ins carry the fields the panel reads.
RobotModelConfig has no init_joints field, so config.init_joints always raised and the branch was dead. Init joints come from the module's _init_joints registry; the preset test now supplies them there.
Replace the bare except Exception: pass / = None handlers around Viser handle attribute writes with warning logs so failures are visible instead of swallowed.
The _urdf / _meshes accesses on viser handles are already confined to single helpers; note the dependency on viser internals so it is not spread elsewhere.
Add a make_panel factory fixture that starts a ViserPanelGui and closes it (stopping its worker threads) during teardown, so cleanup runs even when a test fails mid-body. Convert the GUI tests off hand-rolled try/finally blocks.
github-actions
Bot
added
the
ready-to-merge
github-actions
Bot
added
ready-to-merge
github-actions
Bot
added
ready-to-merge
|
@mustafab0 made some further cleanup and make pink the default ik module. can check again |
# Conflicts: # dimos/manipulation/blueprints.py # docs/capabilities/manipulation/adding_a_custom_arm.md
github-actions
Bot
removed
the
ready-to-merge
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| self._closed = True | ||
| self.state.runtime = PanelRuntime.STOPPING | ||
| self._worker.stop() | ||
| self._operation_worker.stop(timeout=None) |
There was a problem hiding this comment.
close() calls self._operation_worker.stop(timeout=None), which calls thread.join(timeout=None) — an infinite wait. Since the OperationWorker has no operation-level timeout configured (OperationWorker(self._set_error), no timeout_seconds), the worker thread runs each operation to completion before it can be stopped.
Concretely, if the user clicks Plan and then shuts down the module before the planner finishes, ManipulationModule.stop() → world_monitor.stop_all_monitors() → visualization.close() → gui.close() will block here for up to planning_timeout (default 10 s). Likewise, a running preview animation (default 3 s via adapter.preview_path) will hold the shutdown path open for the animation's full duration.
github-actions
Bot
added
the
ready-to-merge
| # Viewer Backends | ||
|
|
||
| Dimos supports Rerun as its visualization backend (`rerun` default, or `none` to disable). | ||
| Dimos stream visualization uses `GlobalConfig.viewer` and `vis_module(...)` to render typed |
There was a problem hiding this comment.
Nitpick: This sentence has become quite hard to parse.
When writing documentation, you should view it from the perspective of someone unfamiliar with the system. It would be more accessible to say:
Dimos uses Rerun for visualizations. It can be disabled by using
dimos --viewer none ....
And then explain what vis_module is for.
(There's no need to mention --viewer rerun, the default. That made sense when we also supported foxglove.)
| Viser support is included in the `manipulation` extra: | ||
|
|
||
| ```bash | ||
| uv sync --extra manipulation |
There was a problem hiding this comment.
| uv sync --extra manipulation | |
| uv sync --extra manipulation --inexact |
| ).transports( | ||
| { | ||
| ("joint_state", JointState): LCMTransport("/coordinator/joint_state", JointState), | ||
| } | ||
| ) | ||
| # The planner's `coordinator_joint_state` input auto-connects to the | ||
| # ControlCoordinator's output on the default `/coordinator_joint_state` | ||
| # topic, so no `.transports(...)` override is needed. | ||
| # The `.transports(...)` override subscribes the planner to the coordinator's | ||
| # published joint-state topic. |
There was a problem hiding this comment.
| ).transports( | |
| { | |
| ("joint_state", JointState): LCMTransport("/coordinator/joint_state", JointState), | |
| } | |
| ) | |
| # The planner's `coordinator_joint_state` input auto-connects to the | |
| # ControlCoordinator's output on the default `/coordinator_joint_state` | |
| # topic, so no `.transports(...)` override is needed. | |
| # The `.transports(...)` override subscribes the planner to the coordinator's | |
| # published joint-state topic. | |
| ) |
Note that I have merged #2515
It renames joint_state to coordinator_joint_state and you rely on defaults, so no need to use .transports(...) here.
| world = MagicMock() | ||
| return WorldMonitor( | ||
| world=world, | ||
| visualization=cast("VisualizationSpec | None", viz), |
There was a problem hiding this comment.
No need for casting. We don't typecheck in tests. But even so, the problem is the bad type in the function definition: viz: object | None.
| kinematics=MagicMock(), | ||
| ) | ||
|
|
||
| with ( |
There was a problem hiding this comment.
This very long with statement is identical to the one above. Use a pytest fixture to remove duplication.
3 participants
Closes #2412 and supersedes #2413
Adds Viser as a manipulation planning visualizer and rewires manipulation visualization config around a single validated
visualizationfield. The new visualizer is more feature-rich than the original drake-bundled meshcat vis and provides a moveit style UI for plan/preview/execute.Main changes:
visualization_backend/visualization_optionsconfig with a Pydantic discriminatedvisualizationconfig fornone,meshcat, andviser.WorldSpecPlanningSpecsVisualizationSpecWorldMonitor.viser[urdf]in themanipulationextra and move Viser usage docs into the manipulation docs.Notes
allow_plan_execute.set_planning_target/clear_planning_targetVisualizationSpecAPI was reverted; target controls remain internal to the Viser GUI/scene.How to Test
Contributor License Agreement