Repository navigation
Conversation
…quid-handling) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lt report Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…on summary Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…idates Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…atings Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Text2WetLab: benchmark analysis and expansion planHow tasks are classifiedEvery task gets three discrete labels instead of a numeric score.
A fourth, expert-labelled tag records why failures happen: bench-ready, needs training (bead handling, magnet timing), Non-liquid-handling candidates have no pipetting structure, so they get a family instead: PL protocol program 1. The 11 current tasks
Current tasks on the grid
✅ solved by all · 2. The new candidatesAll 210 new candidates are listed with a plain-English description in New liquid-handling candidates by structure and readiness
New non-liquid-handling candidates by family and readiness
Which specification levels the new candidates can support
Every candidate with both a paper and code (138) can become a paper-assisted and a paper-derived task, which fills the 3. Where the models go wrongA deterministic diff of each model's run log against the oracle's (
4. Size of the expanded benchmark
Simulator check ( 5. Recommendations
|
… of all 210 candidates Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
All new expansion candidates (plain-English descriptions)210 new candidates found by the four searches, after removing papers already in the corpus. Structure levels: S1 single transfer · S2 plate set-up · S3 wash cycles · S4 module-coordinated · S5 multi-reaction workflow. Ready (44)
Partial (108)
|
All new expansion candidates (continued)No (58)
|
Registry and per-candidate records under sources/experimental/, kept apart from the curated corpus. PDFs and clones stay in the ingest cache outside the repo. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…epos; fix three wrong code links Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tocols, plus 3D replay tools Opentrons 7.5 + PyLabRobot, run with no network and read-only code. 203 of 795 candidate protocols pass; a Hamilton STARlet protocol runs on PyLabRobot's simulated backend. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Which candidate protocols actually run (Docker simulation, 2026-10-13)Every Opentrons protocol in the experimental candidates was simulated in the Tier 1: paper + code + results, simulating (10 repos, the strongest new tasks)
Tier 2: protocols run, weaker paper/results link (17 repos)
Tier 3: run but not tasks (3 repos)BiomationScripter, the Newcastle labware collection and LabOP are libraries or templates with no real liquid moves. HamiltonThe har1eyk STARlet Automated Protein Extraction protocol (PyLabRobot) runs end to end on PyLabRobot's simulated backend: 936 liquid moves, 164 tip pickups and 10 CoRe-gripper moves of the binding plate onto the magnet and back. Reference videosFour 3D replays are attached to a draft release (outside the codebase; drafts are visible only to repository collaborators):
Limits: "passes" means the protocol runs in the simulator, which does not track liquid or check the biology. The two Flex candidates and Culturomics simulate but were not rendered, because the replay only reads OT-2 slot numbers. |
Research branch for growing Text2WetLab beyond its 11 tasks. Nothing here changes the benchmark itself; everything is under
research/expansion/.merged_candidates.jsonland summarised inSUMMARY.mdVERIFIED.md)tools/,fault_report.md)ANALYSIS.md, also posted below as a comment)🤖 Generated with Claude Code