OpenAI Tested AI on 100 Real Lab Tasks. The Hard Problem Is Knowing When Not to Trust It
OpenAI's 6 November BioBasicsBench study tested 100 routine laboratory interpretation tasks. The score gains are real within its rubric, but the business case depends on catching unreliable results.
Wealth / The Compound journalPhoto: Gannu03 / Wikimedia Commons; CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/); 11 December 2025; adapted crop · Illustrative image.
The next commercial test for scientific AI is not whether it can describe a laboratory image. It is whether it knows when the evidence is too weak to justify an experimental decision. OpenAI’s BioBasicsBench study, released on 6 November 2026, is useful precisely because it examines routine, messy lab evidence rather than polished figures from finished scientific papers.
OpenAI and Ligo developed the evaluation around 100 tasks based on real laboratory experiments. They cover protein methods, imaging and cytometry, quantification and assays, nucleic acids and molecular cloning. The inputs include gel and blot images, microscope readings, plate photographs, instrument exports and short videos. That mixture matters: a misplaced interpretation of a control or a mistaken unit can invalidate subsequent work even when the prose sounds convincing.
This is a research benchmark, not a claim that OpenAI has launched a validated AI laboratory product, and it does not establish clinical, GMP or manufacturing suitability.
What the numbers show — and what they do not
At the highest reported reasoning-effort setting, GPT-6 Astra received 66.7% mean rubric credit, compared with 43.8% for GPT-5.6 Sol. That is a 22.9 percentage-point improvement, or roughly 52% more rubric credit relative to the older score. Those are comparisons of the reported evaluation means, not percentages of experiments fully completed correctly. The underlying tasks can award partial credit for distinct reasoning and measurement criteria.
OpenAI reports seven evaluated models, 37 model-effort configurations and five attempts per task. That makes 18,500 graded responses across the 100 original tasks. Repeated attempts help assess a method’s variability, but they do not turn 100 task scenarios into 18,500 independent experiments. The study also describes task-level confidence intervals rather than pretending every scored answer is an isolated scientific trial.
At the same xhigh setting, Astra averaged 4,704 output tokens per task versus 9,099 for Sol. That is roughly 48% fewer output tokens, not necessarily 48% lower API spend: model pricing, non-output tokens, tool use and retries can change the bill. Any procurement comparison must use the actual price schedule and workflow.
The ambiguity test is more valuable than the headline score
The study includes an example of fluorescence measurements in which a signal could come from more than one dye. A model that simply labels a bright region as dead cells might give a confident but unsupported conclusion. The improved model instead recognised that overlapping detection windows meant the controls could not isolate the cause of the signal.
That behaviour is an important commercial differentiator. A useful lab assistant should be able to ask for a proper negative control, flag a missing replicate, identify a doubtful lane assignment or say a quantitative inference cannot be made from the available record. These actions may appear less impressive than a clean-looking result, but they can prevent downstream experimental waste.
For a biotech operator, the quality target should be an accepted scientific interpretation with its evidence trail. A fast answer that triggers an unnecessary repeat experiment may be worse than a slower method that flags uncertainty.
Why this is a harder business than a writing assistant
A general document assistant operates largely on text. A laboratory workflow can require chain-of-custody information, equipment calibration, sample identifiers, experimental protocols, data files and human observations. An agent must interpret what it sees without silently manufacturing missing facts.
Consider a Western blot with inconsistent loading controls. The correct business outcome might be to reject a comparison, not produce an attractive fold-change estimate. The same principle applies to flow-cytometry gates, sequencing quality and assay standard curves. A model can help with triage and quality checks, but a scientist must still confirm the interpretation and the limits of the method.
For regulated manufacturing and quality control, this benchmark provides no evidence of 21 CFR Part 11 validation, GxP qualification, data integrity controls or a replacement for approved procedures. Those are separate organisational and regulatory requirements. An analysis tool that works on research photographs must not be marketed as a released-batch decision system on that basis alone.
A practical return-on-investment test
An illustrative pilot might take 50 routine data-review jobs that normally require 25 minutes of staff effort each. Suppose a human reviewer still needs 12 minutes per AI-assisted job to verify source data, controls and reasoning. The gross capacity released would be 50 × 13 minutes, or 10.8 hours. If ten jobs also need another ten minutes each of correction, that becomes approximately 9.2 hours before tool, integration, training and audit costs.
These are scenario assumptions, not measured OpenAI customer savings. They also assume every job eventually receives acceptable review. Real outcomes could be worse if sample metadata are incomplete, if the AI produces plausible wrong conclusions or if reviewers must repeat the analysis from scratch.
A credible pilot would log accepted versus rejected outputs, false reassurance on invalid controls, number of escalations, time spent on corrections, actual run costs and percentage of decisions that remain traceable to original evidence. An error severity framework matters because a rare high-consequence failure can outweigh many small time savings.
Where OpenAI might eventually earn money
If evidence interpretation becomes reliable enough, scientific organisations may pay for a governed assistant that triages low-risk tasks and routes uncertain cases to staff. The potential benefit is not merely faster report writing; it is fewer unproductive experiment loops and more consistent review. But scientific purchasing cycles can be longer than consumer software cycles, and integrating with equipment exports or existing laboratory information systems is not trivial.
OpenAI and Ligo did not disclose BioBasicsBench-derived subscription revenue, deployment agreements, accuracy in a GMP setting, integration margins or an independently audited productivity gain. The research is a signal of capability improvement and possible future demand, not a financial forecast. OpenAI remains privately held; this is not an investable quoted-stock event.
The more useful milestone to watch is not a higher percentage on one benchmark, but independent, domain-specific testing that measures safe rejection, repeatability and the cost of human verification. Until then, the sensible use is expert-assisted interpretation, not unattended conclusions.
Read the broader OpenAI private-company profile, or our separate analysis of security at the integration layer for a different challenge facing increasingly capable agents.
Research checked 16 November 2026. The underlying OpenAI study was published on 6 November; all scenario costs are editorial assumptions. Cover: Gannu03, Wikimedia Commons, December 2025, CC BY-SA 4.0; adapted crop, documentary laboratory equipment photograph, not imagery from OpenAI or Ligo.


