[Tyronita]

Evidence for #63, which rendered every task and candidate with opentrons-mujoco-viz. Stacked on research/benchmark-expansion, so the diff is only the analysis. Related: #65 (provenance of linked simulators and datasets).

What is added (all under research/expansion/)

• MUJOCO_VIZ_ANALYSIS.md: source review of opentrons-mujoco-viz at commit 460169f and its measured behaviour.
• tools/probe_mujoco_viz.py and mujoco_viz_probe.json: the measurements and their raw output.
• tools/side_by_side.sh: builds PyLabRobot | MuJoCo videos from the same oracle run log.
• tools/mj_render.py: caption strip and sampling so long protocols render to a fixed length.

Findings

The package draws a recognisable OT-2, keeps correct liquid and module state, and renders a video in about 5 seconds. But the simulated gantry does not go where it is commanded:

Finding	Measurement
Tip misses every slot	17–103 mm from the slot centre, after six extra seconds to settle
Front half of the deck unreachable	y command for slots 1–6 is −0.34 / −0.255 m; joint range is ±0.24 m, so it clamps
Collisions block x	All 82 geoms collide; 12–16 contacts at rest. With contacts off, x tracks exactly
Height barely changes	Safe and working heights differ by 15 mm on screen instead of 150 mm
Wells not targeted	The commanded target for any well is the slot centre
Liquid shown for 8 wells only	Column 1 of the plate in slot 2
Impossible volumes accepted	5,000 µL in the pipette; source wells never tracked
Cameras	4 of 5 named cameras render black, including the default
Log parser	Reads 0 of 24 liquid moves from Opentrons 7.5 output

The document lists nine fixes in order. Once the gantry moves correctly, tip-to-labware contacts become meaningful and a collision or reach check can be added to fault_detect.py.

Videos

Side-by-sides for all 11 tasks are in the private Hugging Face dataset EvanOLeary/text2wetlab-3d-replays (side_by_side/). Four examples are attached to the draft release linked from the comment on #65.

Not established

The cause of the z-axis drift was measured but not isolated. One commit of the package and one MuJoCo version (3.15) were examined.

🤖 Generated with Claude Code

[qte77]

This lines up with what we found independently on the same commit, posted later on opentrons-mujoco-viz#2. #66 is the more complete analysis. A few things it doesn't cover yet, with repro scripts on this branch:

• Camera cause. The camera euler values are in degrees, but ot2.xml sets <compiler angle="radian">. Converting the five camera eulers to radians in memory makes all five render the deck (before/after), so fix 7 is a five-value change.
• z with contacts off. The simulation becomes numerically unstable: MuJoCo warns "Nan, Inf or huge value in QACC … simulation is unstable", and z velocity reaches hundreds of m/s. That fits the −0.517 m past the −0.38 m limit in finding 2. Untested guess at the cause: act_z with kv=400 on the 0.2 kg z body is too stiff for explicit RK4 at timestep=0.002; integrator="implicitfast" would be the first thing to try.
• Parser lines beyond liquid moves, from the Opentrons v7.5.0 message templates (not a real log): mix prints ul, not uL; thermocycler block temperature (well block temperature), the profile (Thermocycler starting N repetitions …) and all heater-shaker lines don't match; magnet engage has no height in its text, so it always falls back to 13 mm. If the eval/runlog.py pattern is reused with the package's _slot(), a number in the labware name wins (Agilent 1 Well Reservoir … on slot 3 gives slot 1). Taking the number after the final on / on slot handles both 7.5 (on 1) and 8.x (on slot 3).
• Silent failure. render_protocol_from_simulate never checks the simulator's returncode, so a failed run renders an empty video.

Three questions, so we work in your order:

1. Fix 8: update the package's parser, or replace it with eval/runlog.py? We announced a parser PR on opentrons-mujoco-viz#2 and are happy to do either.
2. Shall we take fixes 1, 2 and 7 (collision groups, y range, cameras) as separate small PRs in opentrons-mujoco-viz, in your order?
3. Could the 27-line Opentrons 7.5 log behind finding 8 be committed, here or in opentrons-mujoco-viz, as a test fixture?

Generated with Claude noreply@anthropic.com