Hopp til hovedinnhold

OpenAI tests AI agents on wet-lab data: Astra scores 66.7% — still unreliable

OpenAI and Ligo launch BioBasicsBench: 100 wet-lab tasks on raw data. GPT‑6 Astra scores 66.7% at high effort versus 43.8% for GPT‑5.6 Sol — still not reliable enough for autonomous lab decisions.

Av Joachim HøgbyKilde: OpenAI

OpenAI is launching BioBasicsBench, a new test of whether AI agents can handle everyday wet-lab work: interpreting raw experimental data, spotting errors, and admitting when the evidence cannot support a conclusion. GPT‑6 Astra scores 66.7% at high reasoning effort, versus 43.8% for GPT‑5.6 Sol.

The announcement came via OpenAI’s official blog and X account on Friday 6 November 2026. The benchmark was built with Ligo, a biotech company focused on enzyme design, and draws on 100 tasks from real laboratory experiments.

Raw evidence, not polished figures

According to OpenAI, existing benchmarks often rely on charts and images already selected for scientific papers. BioBasicsBench is meant to mirror what scientists face daily: failed runs, ambiguous images and conflicting observations. Tasks span five areas — protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning — from reading molecular weight on a gel to detailed assay analysis.

Agents receive raw evidence such as gel and blot images, colony plates, microscopy, instrument exports, sequencing files and bench videos. Answers are scored against expert rubrics so partial analyses can be distinguished from full credit. OpenAI says each task was reviewed internally by a Ligo scientist, externally by a PhD-level wet-lab scientist, then reviewed internally again.

Astra improves — still unreliable

OpenAI evaluated seven models across 37 reasoning-effort settings, with five attempts per task (18,500 results in total). At the “xhigh” setting, GPT‑6 Astra earned a mean 66.7% of available credit (95% CI 60.3–73.1), versus 43.8% for GPT‑5.6 Sol. OpenAI says stronger multimodal capability appears to drive much of the gain.

At the same time, OpenAI’s own failure cases show Astra can still let assumptions override controls, discount contradictory observations and miscount colonies. Higher benchmark scores, the company stresses, do not mean models are ready to make reliable lab judgments.

Why it matters for leaders

  • Lab AI is a measurable product requirement, not a demo. If vendors promise “AI that reads lab results”, ask for concrete scores on raw data — not paper figures — and whether models flag when evidence is insufficient.
  • Wrong conclusions burn R&D. Misreading controls or images can send teams down expensive blind alleys. For biotech, pharma and medtech this is a quality-system issue, not only a productivity play.
  • Biosafety travels with capability. OpenAI explicitly links better wet-lab skills to the need to track biosafety and biosecurity implications. Leaders rolling out lab agents need policy for which tasks are allowed and who signs off conclusions.
  • Humans stay in the loop. Even Astra at top effort sits far below what an experienced wet-lab scientist is expected to master. Use agents for triage and quality checks, not as autonomous decision-makers.

Sources and media

Illustration: AI-generated with Grok Bot / hogby.ai

Mer om AI

Oppskriften. Strategien. Hver dag.

AI-nyheter for ledere. Kuratert av en CIO som bygger det selv. Daglig i innboksen.