原帖引用 · 原文
Rohan Paul@rohanpaul_ai
OpenAI built BioBasicsBench, a test that gives AI agents the messy raw data of real lab work, like gel photos and microscope videos, and grades whether they read it correctly.
GPT-6 Astra earned 66.7% of available credit on a new wet-lab benchmark, versus 43.8% for GPT-5.6 Sol.
BioBasicsBench, built with enzyme-design company Ligo, draws 100 tasks from scientists’ raw gels, colony plates, microscopy, sequencing reads and bench videos.
Its rubrics give partial credit, including for spotting errors or saying evidence can’t support conclusions.
At the xhigh setting, Astra used 4,704 output tokens per task, about half the older model’s, so longer reasoning didn’t drive the gain.
A new way to measure how AI agents handle the routine analyses of wet-lab science—BioBasicsBench.
Developed with Ligo, a biotech company focused on enzyme design, the benchmark draws on 100 tasks from real experiments to test whether agents can interpret raw lab data, identify errors, and recognize when the evidence doesn’t support a conclusion.
Accelerating biological research starts with getting the everyday work right. BioBasicsBench shows where models need to improve, and helps us track what those gains mean for scientific progress, biosafety, and biosecurity.
https://t.co/wN2xLk0YeW
在 X 查看这条帖子 ↗
· UTC+8