AI
A cloning benchmark shows frontier AI still struggles with complete lab plans
BenchBench-Cloning evaluates whether language models can design molecular constructs and produce executable cloning protocols. Its 65 tasks draw on sequence records, laboratory notebooks and published studies, with protocol plans tested in a cloning simulator and construct designs checked deterministically or by rubric. Across eight systems, the top reported score is 30.6%, while others range from 8.6% to 22.2%; even repeated best-of-ten sampling leaves the benchmark unsaturated. The results indicate that integrating many individually plausible laboratory decisions remains difficult.
Editorial summary · Publisher article
What matters
- The benchmark contains 50 protocol-development tasks and 15 construct-design tasks.
- Plans are tested through simulation and task-specific construct checks.
- The highest reported system score is 30.6%, with substantial room for improvement.
Context & caveats
- The supplied abstract does not establish how simulated performance transfers to wet-lab success.
Why this made the edition
The benchmark measures end-to-end experimental planning rather than isolated factual recall and reports clear performance differences across eight systems.
Source material