diff --git a/research/expansion/MUJOCO_VIZ_ANALYSIS.md b/research/expansion/MUJOCO_VIZ_ANALYSIS.md new file mode 100644 index 0000000..1173ab0 --- /dev/null +++ b/research/expansion/MUJOCO_VIZ_ANALYSIS.md @@ -0,0 +1,141 @@ +# opentrons-mujoco-viz: source analysis and measured behaviour + +Reviewed: `PhysicalAIBenchmarks/opentrons-mujoco-viz` at commit `460169f` (about 1,275 lines: `scene.py` 352, +`event_driver.py` 249, `generate_labware.py` 197, `modules.py` 180, `assets/ot2.xml` 246), with MuJoCo 3.15. +Every number below was measured by `tools/probe_mujoco_viz.py`; raw output is in `mujoco_viz_probe.json`. +Evidence for PR #63, which rendered all tasks and candidates with this package. + +## Summary + +The package draws a recognisable OT-2 and keeps correct liquid and module state, and it renders a video in about 5 seconds. +But the simulated gantry does not go where it is commanded: the pipette tip ends 17–103 mm from the centre of every slot, +never changes height, and never visits an individual well. The MuJoCo part therefore adds motion that looks plausible but is +not a faithful account of the protocol, and it cannot yet support collision or reach checks. + +## How it works + +| Part | What it does | +|---|---| +| `assets/ot2.xml` | A hand-built scene of 17 bodies and 82 box/cylinder geoms, no meshes. One fixed deck: tip rack (slot 1), plate (2), trough (3), magnetic module (4), temperature module (6), thermocycler (7/8), heater-shaker (9), trash (12). Three slide joints (x, y, z) with position actuators, plus the thermocycler lid. | +| `modules.py` | `ProtocolState`: pure-Python bookkeeping of pipette volume, tip, per-well volumes and module states, updated by each event. The docstring in `scene.py` says so directly: "Liquid volumes are tracked analytically; MuJoCo handles gantry kinematics only." | +| `scene.py` | `WetLabScene.apply_event` moves the gantry to the event's slot for a fixed number of physics steps, updates the state, recolours geoms, and returns frames. `render_with_hud` overlays step count, tip, pipette volume and module status. | +| `event_driver.py` | Regex parser for `opentrons_simulate` text output and a `render_protocol` wrapper. | +| `generate_labware.py` | Builds OBJ meshes from Opentrons labware definitions. Not used by the bundled scene (0 meshes loaded). | + +## Findings + +### 1. The gantry does not reach the commanded position + +Tip position after `move_to_slot(slot, work=True)` plus six further seconds of simulated time: + +| Slots | Row | Tip distance from slot centre | +|---|---|---| +| 1, 2, 3 | front | 102–103 mm | +| 4, 5, 6 | second | 23–44 mm | +| 7, 8, 9 | third | 18–45 mm | +| 10, 11, 12 | back | 17–40 mm | + +Three separate causes: + +- **Y is out of range for the front half of the deck.** `_slot_ctrl_y` commands `slot_y − 0.17`, which is −0.34 m for slots 1–3 + and −0.255 m for slots 4–6. The joint and actuator range is ±0.24 m, so the command is clamped and the tip stops at y = −0.07 m + for all six of those slots. The pipette never reaches the front row. +- **X stops 17–45 mm short.** With contacts disabled the x joint reaches its target exactly (−0.152 commanded, −0.152 achieved), + so the shortfall comes from collisions. +- **Motion is not finished when frames are taken.** `apply_event` allows 100 physics steps (0.2 s) per move. After that budget the + x joint was still at −0.013 m for a −0.152 m command (slot 1) and at −0.121 m for a 0 m command (slot 2), so rendered frames show + the gantry part-way to, or still leaving, the previous slot. + +### 2. The pipette never changes height + +`SAFE_Z` is −0.02 m and `WORK_Z` is −0.17 m, but the z joint sits at −0.354 m and −0.369 m respectively, and the tip is at +z = −0.152 m over every slot. The two heights differ by 15 mm on screen instead of 150 mm. The actuator is pushing with about +2,700 N against 16 active contacts, and turning gravity off does not free it. With contacts off as well the joint goes to −0.517 m, +beyond its stated −0.38 m limit, so the limit is not being enforced either. The cause of the downward drift was not isolated. + +### 3. Collisions are on for every geom, and they are the wrong ones + +All 82 geoms have collision enabled and there are 12–16 active contacts at all times, including at rest. These are the model's own +parts touching each other, not the tip touching labware. Until they are removed, contact counts say nothing about a protocol. + +### 4. Wells are not targeted + +`move_to_slot` accepts a `well_offset_x`, but `apply_event` never passes one and there is no y offset at all, so the commanded +target for well A1 and well H12 of the same plate is the same point, the slot centre. Every operation on a plate is aimed at one spot. +(This is read from the code; the measured joint positions for the two wells differ only because the gantry had not settled.) + +### 5. Only eight wells show liquid + +The state tracks volume for any slot and well (100 µL dispensed into slot 5 C7 is recorded), but the scene only recolours eight +geoms, `w_A1` to `w_H1` in slot 2. Liquid anywhere else on the deck is invisible. This is why the four panels of a +model-versus-oracle grid look the same. + +### 6. Liquid bookkeeping accepts impossible volumes + +- Aspirating 5,000 µL leaves 5,000 µL in the pipette; `PIPETTE_MAX = 1000.0` is defined but never checked. +- Wells start at 0 and `add_to_well` clamps at 0, so aspirating from a source leaves it at 0. Source volumes are never tracked and + drawing from an empty well is not detected. +- The tip-liquid indicator saturates at 200 µL regardless of pipette size. + +### 7. Four of the five named cameras render black + +Mean pixel brightness (0–255): `iso` 0.0, `side` 0.0, `front` 0.0, `deck` 0.03, `top` 87.8, and the built-in fallback free camera 31.0. +`render()` and `apply_event()` default to `iso`, so the documented quick start produces black frames. `scripts/render_run.py` +in this repository already works around this by forcing the fallback camera. + +### 8. The package's parser does not read current Opentrons output + +On the `a1-a12-100ul` reference protocol under Opentrons 7.5, the log has 12 aspirate and 12 dispense lines; `parse_simulate_output` +returns 2 events (one pick, one drop) and no liquid moves. Its pattern expects `well A1 of on `; Opentrons 7.5 prints +`Aspirating 100.0 uL from A1 of reservoir on 1 at 92.86 uL/sec`. `render_run.py` and `tools/mj_render.py` avoid it by converting +events from `eval/runlog.py`. + +### 9. The deck is the same for every protocol + +The scene has no notion of what a protocol loaded. A protocol with tube racks in slots 1 and 2 is drawn with a tip rack and a +96-well plate there. `generate_labware.py` could supply real geometry but is not wired in. + +## What works + +- State tracking of pipette, tip, well volumes and module states is straightforward and correct for well-formed logs. +- Module feedback: magnet bar and LED, temperature LED colour, thermocycler lid actuation, heater-shaker latch. +- Speed: about 5 s for a 45 s video, headless, with no browser or GPU. +- The HUD shows step count, tip state, pipette volume and elapsed protocol time. + +## Compared with the PyLabRobot replay + +| | opentrons-mujoco-viz | PyLabRobot replay (`tools/replay_log.py`) | +|---|---|---| +| Deck | One fixed layout | Built from the labware the protocol loaded | +| Position shown for an operation | 17–103 mm from the slot centre; wells not distinguished | The exact well, from the labware definition | +| Height changes | 15 mm instead of 150 mm | Marker above the well (no z motion modelled) | +| Liquid shown | Eight wells of one plate | Every well and tube | +| Tips | One indicator | Every tip position | +| Faults | None | Marked by `fault_detect.py` | +| Robots | OT-2 | OT-2, and a Hamilton STARlet | +| Physics | Rigid-body dynamics, currently obstructing the motion | None | +| Render time per video | About 5 s | 1–2 min, needs a GPU browser | + +Side-by-side videos of all 11 tasks, both driven by the same oracle run log, are in the private Hugging Face dataset +`EvanOLeary/text2wetlab-3d-replays` under `side_by_side/` (built by `tools/side_by_side.sh`). + +## What to fix, in order + +1. Turn collisions off for the gantry and decorative geoms (`contype="0" conaffinity="0"`), or add contact exclusions, so the + gantry can move. This alone should fix x tracking. +2. Correct the y mapping or widen the y range so slots 1–6 are reachable. +3. Find why z does not hold its commanded height, and enforce the joint limit. +4. Step until the position error is below a tolerance instead of a fixed 100 steps. +5. Pass per-well offsets from the labware definition into `move_to_slot`. +6. Build the deck from the protocol's labware (wire in `generate_labware.py`) and recolour every well. +7. Fix the named cameras and make a working one the default. +8. Update the parser to the Opentrons 7.x log format, or drop it in favour of `eval/runlog.py`. +9. Enforce pipette capacity and allow starting volumes. + +Once 1–5 hold, contacts between the tip and labware become meaningful, and a collision or reach check can be added to +`fault_detect.py` as a new fault type. That is the capability MuJoCo offers that the PyLabRobot replay cannot. + +## Limits of this analysis + +One commit of the package was examined with one MuJoCo version. The z-axis behaviour was measured but its cause was not isolated. +The tip position is the centre of the `pip_tip` geom; slot centres are the package's own `SLOT_XY` table. diff --git a/research/expansion/mujoco_viz_probe.json b/research/expansion/mujoco_viz_probe.json new file mode 100644 index 0000000..c969d5c --- /dev/null +++ b/research/expansion/mujoco_viz_probe.json @@ -0,0 +1,451 @@ +{ + "model": { + "bodies": 17, + "geoms": 82, + "joints": 4, + "actuators": 4, + "cameras": 5, + "meshes": 0, + "timestep_s": 0.002, + "geoms_with_collision_enabled": 82 + }, + "joint_ranges_m": { + "j_tc_lid": [ + 0.0, + 0.06 + ], + "j_x": [ + -0.29, + 0.29 + ], + "j_y": [ + -0.24, + 0.24 + ], + "j_z": [ + -0.38, + 0.02 + ] + }, + "camera_mean_brightness_0_255": { + "iso": 0.0, + "side": 0.0, + "top": 87.83, + "front": 0.0, + "deck": 0.03, + "free (fallback)": 30.95 + }, + "gantry_by_slot": { + "1": { + "commanded_joint_xyz": [ + -0.152, + -0.34, + -0.17 + ], + "joint_xyz_after_apply_event_budget": [ + -0.0127, + -0.1546, + -0.371 + ], + "joint_xyz_after_6s_more": [ + -0.13, + -0.24, + -0.3688 + ], + "slot_centre_xy": [ + -0.152, + -0.17 + ], + "tip_xy": [ + -0.13, + -0.07 + ], + "tip_error_mm": 102.4, + "tip_z": -0.152, + "contacts": 12 + }, + "2": { + "commanded_joint_xyz": [ + 0.0, + -0.34, + -0.17 + ], + "joint_xyz_after_apply_event_budget": [ + -0.1214, + -0.24, + -0.3685 + ], + "joint_xyz_after_6s_more": [ + -0.019, + -0.24, + -0.3688 + ], + "slot_centre_xy": [ + 0.0, + -0.17 + ], + "tip_xy": [ + -0.019, + -0.07 + ], + "tip_error_mm": 101.8, + "tip_z": -0.152, + "contacts": 12 + }, + "3": { + "commanded_joint_xyz": [ + 0.152, + -0.34, + -0.17 + ], + "joint_xyz_after_apply_event_budget": [ + -0.0014, + -0.24, + -0.3685 + ], + "joint_xyz_after_6s_more": [ + 0.1279, + -0.24, + -0.3688 + ], + "slot_centre_xy": [ + 0.152, + -0.17 + ], + "tip_xy": [ + 0.128, + -0.07 + ], + "tip_error_mm": 102.8, + "tip_z": -0.152, + "contacts": 12 + }, + "4": { + "commanded_joint_xyz": [ + -0.152, + -0.255, + -0.17 + ], + "joint_xyz_after_apply_event_budget": [ + 0.1064, + -0.24, + -0.3687 + ], + "joint_xyz_after_6s_more": [ + -0.1115, + -0.24, + -0.3688 + ], + "slot_centre_xy": [ + -0.152, + -0.085 + ], + "tip_xy": [ + -0.111, + -0.07 + ], + "tip_error_mm": 43.7, + "tip_z": -0.152, + "contacts": 12 + }, + "5": { + "commanded_joint_xyz": [ + 0.0, + -0.255, + -0.17 + ], + "joint_xyz_after_apply_event_budget": [ + -0.1069, + -0.24, + -0.3685 + ], + "joint_xyz_after_6s_more": [ + -0.0168, + -0.24, + -0.3688 + ], + "slot_centre_xy": [ + 0.0, + -0.085 + ], + "tip_xy": [ + -0.017, + -0.07 + ], + "tip_error_mm": 22.7, + "tip_z": -0.152, + "contacts": 12 + }, + "6": { + "commanded_joint_xyz": [ + 0.152, + -0.255, + -0.17 + ], + "joint_xyz_after_apply_event_budget": [ + 0.0003, + -0.24, + -0.3685 + ], + "joint_xyz_after_6s_more": [ + 0.1282, + -0.24, + -0.3688 + ], + "slot_centre_xy": [ + 0.152, + -0.085 + ], + "tip_xy": [ + 0.128, + -0.07 + ], + "tip_error_mm": 28.3, + "tip_z": -0.152, + "contacts": 12 + }, + "7": { + "commanded_joint_xyz": [ + -0.152, + -0.17, + -0.17 + ], + "joint_xyz_after_apply_event_budget": [ + 0.1072, + -0.2086, + -0.3698 + ], + "joint_xyz_after_6s_more": [ + -0.1075, + -0.17, + -0.3708 + ], + "slot_centre_xy": [ + -0.152, + 0.0 + ], + "tip_xy": [ + -0.107, + -0.0 + ], + "tip_error_mm": 45.0, + "tip_z": -0.154, + "contacts": 16 + }, + "8": { + "commanded_joint_xyz": [ + 0.0, + -0.17, + -0.17 + ], + "joint_xyz_after_apply_event_budget": [ + -0.104, + -0.17, + -0.3706 + ], + "joint_xyz_after_6s_more": [ + -0.018, + -0.17, + -0.3708 + ], + "slot_centre_xy": [ + 0.0, + 0.0 + ], + "tip_xy": [ + -0.018, + -0.0 + ], + "tip_error_mm": 18.0, + "tip_z": -0.154, + "contacts": 16 + }, + "9": { + "commanded_joint_xyz": [ + 0.152, + -0.17, + -0.17 + ], + "joint_xyz_after_apply_event_budget": [ + -0.0009, + -0.17, + -0.3706 + ], + "joint_xyz_after_6s_more": [ + 0.1264, + -0.17, + -0.3693 + ], + "slot_centre_xy": [ + 0.152, + 0.0 + ], + "tip_xy": [ + 0.126, + -0.0 + ], + "tip_error_mm": 26.0, + "tip_z": -0.152, + "contacts": 16 + }, + "10": { + "commanded_joint_xyz": [ + -0.152, + -0.085, + -0.17 + ], + "joint_xyz_after_apply_event_budget": [ + 0.1055, + -0.1247, + -0.37 + ], + "joint_xyz_after_6s_more": [ + -0.1116, + -0.085, + -0.3688 + ], + "slot_centre_xy": [ + -0.152, + 0.085 + ], + "tip_xy": [ + -0.112, + 0.085 + ], + "tip_error_mm": 40.0, + "tip_z": -0.152, + "contacts": 12 + }, + "11": { + "commanded_joint_xyz": [ + 0.0, + -0.085, + -0.17 + ], + "joint_xyz_after_apply_event_budget": [ + -0.107, + -0.085, + -0.3685 + ], + "joint_xyz_after_6s_more": [ + -0.0168, + -0.085, + -0.3688 + ], + "slot_centre_xy": [ + 0.0, + 0.085 + ], + "tip_xy": [ + -0.017, + 0.085 + ], + "tip_error_mm": 17.0, + "tip_z": -0.152, + "contacts": 12 + }, + "12": { + "commanded_joint_xyz": [ + 0.152, + -0.085, + -0.17 + ], + "joint_xyz_after_apply_event_budget": [ + 0.0003, + -0.085, + -0.3685 + ], + "joint_xyz_after_6s_more": [ + 0.1271, + -0.085, + -0.3693 + ], + "slot_centre_xy": [ + 0.152, + 0.085 + ], + "tip_xy": [ + 0.127, + 0.085 + ], + "tip_error_mm": 25.0, + "tip_z": -0.152, + "contacts": 16 + } + }, + "joint_xyz_for_two_wells_in_slot_2": { + "A1": [ + 0.0959, + -0.2149, + -0.3695 + ], + "H12": [ + 0.0658, + -0.2398, + -0.3684 + ] + }, + "what_stops_the_gantry": { + "z_column_mass_kg": 0.2, + "SAFE_Z": { + "commanded": -0.02, + "j_z": -0.3544, + "tip_z": -0.1374, + "contacts": 16, + "actuator_force_N": 2674.9 + }, + "WORK_Z": { + "commanded": -0.17, + "j_z": -0.3693, + "tip_z": -0.1523, + "contacts": 16, + "actuator_force_N": 1594.6 + }, + "SAFE_Z, gravity off": { + "commanded": -0.02, + "j_z": -0.3618, + "contacts": 16 + }, + "contacts off": { + "commanded_xyz": [ + -0.152, + -0.17, + -0.02 + ], + "joint_xyz": [ + -0.152, + -0.17, + -0.5169 + ] + }, + "contacts off, x after the 100-step budget": { + "commanded": 0.152, + "j_x": 0.1421 + } + }, + "liquid_bookkeeping": { + "pipette_uL_after_aspirating_5000": 5000.0, + "source_well_uL_after_aspirating_from_empty": 0.0, + "state_tracks_slot5_C7_uL": 100.0, + "well_geoms_that_change_colour": [ + "w_A1", + "w_B1", + "w_C1", + "w_D1", + "w_E1", + "w_F1", + "w_G1", + "w_H1" + ] + }, + "own_parser_on_opentrons_7_5": { + "log_lines": 27, + "aspirate_lines": 12, + "dispense_lines": 12, + "events_parsed": { + "pick": 1, + "drop": 1 + }, + "sample_line": "Aspirating 100.0 uL from A1 of reservoir on 1 at 92.86 uL/sec" + } +} diff --git a/research/expansion/tools/probe_mujoco_viz.py b/research/expansion/tools/probe_mujoco_viz.py new file mode 100644 index 0000000..a894270 --- /dev/null +++ b/research/expansion/tools/probe_mujoco_viz.py @@ -0,0 +1,82 @@ +"""Empirical checks of opentrons-mujoco-viz (run with the interpreter that has it and mujoco installed). + + python probe_mujoco_viz.py > mujoco_viz_probe.json + +Measures rather than reads: which cameras render, whether the gantry reaches the position it is commanded to +(by joint name, within the step budget apply_event uses and after six more seconds), what stops it (gravity and +contacts toggled), what the liquid bookkeeping accepts, and what the package's own log parser gets from real +Opentrons output. +""" +import json, subprocess, sys +import mujoco +from opentrons.visualization import WetLabScene, parse_simulate_output +from opentrons.visualization.scene import SLOT_XY, WORK_Z, SAFE_Z + +out = {} +with WetLabScene() as s: + m, d = s.model, s.data + q = lambda n: round(float(d.qpos[m.jnt_qposadr[mujoco.mj_name2id(m, mujoco.mjtObj.mjOBJ_JOINT, n)]]), 4) + tip = mujoco.mj_name2id(m, mujoco.mjtObj.mjOBJ_GEOM, "pip_tip") + out["model"] = dict(bodies=int(m.nbody), geoms=int(m.ngeom), joints=int(m.njnt), actuators=int(m.nu), cameras=int(m.ncam), + meshes=int(m.nmesh), timestep_s=float(m.opt.timestep), + geoms_with_collision_enabled=sum(1 for i in range(m.ngeom) if m.geom_contype[i] or m.geom_conaffinity[i])) + out["joint_ranges_m"] = {mujoco.mj_id2name(m, mujoco.mjtObj.mjOBJ_JOINT, i): [round(float(x), 3) for x in m.jnt_range[i]] for i in range(m.njnt)} + cams = {mujoco.mj_id2name(m, mujoco.mjtObj.mjOBJ_CAMERA, i): round(float(s.render(mujoco.mj_id2name(m, mujoco.mjtObj.mjOBJ_CAMERA, i)).mean()), 2) + for i in range(m.ncam)} + cams["free (fallback)"] = round(float(s.render("free").mean()), 2) + out["camera_mean_brightness_0_255"] = cams + + rows = {} + for slot in range(1, 13): + sx, sy = SLOT_XY[slot] + s.move_to_slot(slot, work=True, steps=100) + budget = [q("j_x"), q("j_y"), q("j_z")] + for _ in range(3000): + mujoco.mj_step(m, d) + tx, ty, tz = (round(float(v), 3) for v in d.geom_xpos[tip]) + rows[slot] = dict(commanded_joint_xyz=[round(sx, 3), round(sy - 0.17, 3), WORK_Z], joint_xyz_after_apply_event_budget=budget, + joint_xyz_after_6s_more=[q("j_x"), q("j_y"), q("j_z")], slot_centre_xy=[sx, sy], tip_xy=[tx, ty], + tip_error_mm=round(1000 * ((tx - sx) ** 2 + (ty - sy) ** 2) ** 0.5, 1), tip_z=tz, contacts=int(d.ncon)) + s._move_to(d.ctrl[s._ax], d.ctrl[s._ay], SAFE_Z, steps=200) + out["gantry_by_slot"] = rows + + wells = {} + for w in ("A1", "H12"): + s.apply_event({"type": "dispense", "volume": 10, "slot": 2, "well": w}) + s.move_to_slot(2, work=True) + wells[w] = [q("j_x"), q("j_y"), q("j_z")] + out["joint_xyz_for_two_wells_in_slot_2"] = wells + + zb = mujoco.mj_name2id(m, mujoco.mjtObj.mjOBJ_BODY, "z_column") + z = {"z_column_mass_kg": round(float(m.body_subtreemass[zb]), 3)} + for name, target in (("SAFE_Z", SAFE_Z), ("WORK_Z", WORK_Z)): + s._move_to(0.152, -0.17, target, steps=2000) + z[name] = dict(commanded=target, j_z=q("j_z"), tip_z=round(float(d.geom_xpos[tip][2]), 4), contacts=int(d.ncon), + actuator_force_N=round(float(d.actuator_force[s._az]), 1)) + m.opt.gravity[:] = 0 + s._move_to(0.152, -0.17, SAFE_Z, steps=2000) + z["SAFE_Z, gravity off"] = dict(commanded=SAFE_Z, j_z=q("j_z"), contacts=int(d.ncon)) + m.opt.disableflags |= mujoco.mjtDisableBit.mjDSBL_CONTACT + s._move_to(-0.152, -0.17, SAFE_Z, steps=2000) + z["contacts off"] = dict(commanded_xyz=[-0.152, -0.17, SAFE_Z], joint_xyz=[q("j_x"), q("j_y"), q("j_z")]) + s._move_to(0.152, -0.17, SAFE_Z, steps=100) + z["contacts off, x after the 100-step budget"] = dict(commanded=0.152, j_x=q("j_x")) + out["what_stops_the_gantry"] = z + + s.reset() + s.state.apply_event({"type": "aspirate", "volume": 5000, "slot": 3, "well": "A1"}) + liquid = {"pipette_uL_after_aspirating_5000": s.state.pipette_vol, "source_well_uL_after_aspirating_from_empty": s.state.well_vol(3, "A1")} + s.state.apply_event({"type": "dispense", "volume": 100, "slot": 5, "well": "C7"}) + liquid["state_tracks_slot5_C7_uL"] = s.state.well_vol(5, "C7") + liquid["well_geoms_that_change_colour"] = sorted(s._g_wells) + out["liquid_bookkeeping"] = liquid + +r = subprocess.run([sys.argv[1], sys.argv[2]], capture_output=True, text=True) +lines = [l for l in r.stdout.splitlines() if l.strip()] +kinds = {} +for e in parse_simulate_output(r.stdout): + kinds[e["type"]] = kinds.get(e["type"], 0) + 1 +out["own_parser_on_opentrons_7_5"] = dict(log_lines=len(lines), aspirate_lines=sum("Aspirating" in l for l in lines), + dispense_lines=sum("Dispensing" in l for l in lines), events_parsed=kinds, + sample_line=next((l.strip() for l in lines if "Aspirating" in l), "")) +print(json.dumps(out, indent=1)) diff --git a/research/expansion/tools/side_by_side.sh b/research/expansion/tools/side_by_side.sh new file mode 100755 index 0000000..d722643 --- /dev/null +++ b/research/expansion/tools/side_by_side.sh @@ -0,0 +1,22 @@ +#!/bin/bash +# Side-by-side PyLabRobot | MuJoCo for every task, both driven by the same oracle run log. +S=/private/tmp/claude-501/-Users-nialloleary/36017c47-22e1-459a-91a2-0b4a5634f844/scratchpad +L=$S/plr/logs; O=~/Desktop/Text2WetLab_SideBySide; mkdir -p $O $S/sbs +cd $S/plr +for t in colony-pcr-screening-hard ecoli-heat-shock-transformation-hard golden-gate-assembly-hard opentrons-rna-extraction-hard; do + [ -s cmp/$t.oracle.mp4 ] && continue + per=$(python3 -c " +import json; e=json.load(open('logs/$t.oracle.json'))['events'] +n=sum(1 for x in e if x['kind'] in ('pick','drop','aspirate','dispense')); print(min(0.35, max(0.004, 55.0/max(n,1))))") + venv/bin/python replay_log.py logs/$t.oracle.json $t "Oracle (reference)" $per cmp/$t.oracle.mp4 cmp/$t.oracle.state.json < /dev/null 2>&1 | grep -v "^viewer" | tail -1 +done +for f in $L/*.oracle.json; do + t=$(basename $f .oracle.json); p=$S/plr/cmp/$t.oracle.mp4 + [ -s "$p" ] || { echo "no PLR video for $t"; continue; } + dur=$(ffprobe -v error -show_entries format=duration -of csv=p=0 $p) + (cd $S/mj && venv/bin/python mj_render.py $f $S/sbs/$t.mj.mp4 $dur "MuJoCo" < /dev/null > /dev/null 2>&1) + ffmpeg -nostdin -loglevel error -y -i $p -i $S/sbs/$t.mj.mp4 \ + -filter_complex "[0:v]scale=960:540,tpad=stop_mode=clone:stop_duration=600[a];[1:v]scale=960:540,tpad=stop_mode=clone:stop_duration=600[b];[a][b]hstack=inputs=2[v]" \ + -map "[v]" -t $dur -c:v libx264 -pix_fmt yuv420p -crf 23 $O/$t.mp4 && echo "side-by-side $t ($dur s)" +done +ls $O | wc -l