Before You Trust an AI Scientific Review, Read Its Error Report

TL;DR: Assess an AI scientific review with six error measures, an independent reference standard and an account of what remains untested.

A fluent AI scientific review still needs an error report. Clear writing alone cannot establish reliable evidence retrieval, extraction or interpretation.

Why this matters: a scientific head, research-methods lead or consortium coordinator needs testable error measures before relying on a scientific AI workflow.

The writing may be clear. The citations may look convincing. The conclusion may sound balanced. None of those qualities establishes that the system found the relevant studies, separated independent experiments, copied numerical results correctly or recognized an untested causal assumption.

The real question is: what kinds of errors does the system make, how often does it make them, and can another team reproduce the result?

That is what buyers, researchers and grant consortia should ask before relying on AI-assisted scientific diligence.

One report contains several different tasks

A scientific review is not a single act of reading. It typically involves finding evidence, screening it, linking records of the same study, extracting facts, checking contradictions, judging study limitations and drawing an appropriately narrow conclusion.

An AI tool can be good at one step and poor at another.

A 2026 living rapid review and evidence map (S1) of 134 studies of AI in biomedical evidence synthesis found sharply different performance across tasks. In the heterogeneous studies it summarized, search-automation tools had a median recall of 17%, while large language models used for data extraction had a median accuracy of 63%. These are task-specific medians across heterogeneous evaluation settings, not estimates for Biodius and not universal accuracy rates for every model or workflow. The review reports that human-in-the-loop approaches improved search performance. The authors concluded that no available tool could replace human expertise across the full process.

The point is not “AI cannot help.” The point is that an efficient screening assistant is not automatically a reliable data extractor, and a good extractor is not automatically a trustworthy causal analyst.

Start with a real reference answer

To test a research method fairly, you need a reference set established without letting the method being tested define the answer.

That might include a carefully selected set of public studies whose underlying identities, populations, methods, results and known corrections have been independently checked by qualified reviewers.

Then define what counts as an error before looking at the tool's output.

Otherwise, a system can be tuned to mistakes discovered in the test set and appear to improve, while telling us little about how it would behave on genuinely new problems.

A credible evaluation needs both:

• a development set, where methods may be designed and debugged; and
• a held-out evaluation set, where methods and grading rules are fixed before the results are inspected.

The final report must disclose any prior familiarity with those studies. Replaying the same well-known examples is useful engineering regression testing. It is not independent validation.

Six measurements that matter more than an eloquent answer

The following measurements and evaluation card are our proposed editorial framework. They are not a completed Biodius benchmark or a validated scoring instrument.

1. Study retrieval recall. Of all studies in the independently established eligible set, how many did the workflow actually find? Record misses, not just the number retrieved.

2. Study identity and overlap. Did it join a journal paper, registry entry, abstract and follow-up analysis from one study? Did it mistakenly call two overlapping cohorts independent? This is a concrete, testable provenance problem.

3. Extraction accuracy. Did it get the right denominators, treatment groups, units, time points and uncertainty measures? A correct headline with the wrong safety population is still an error.

4. Unsupported assertions. Among the biological and clinical statements the system made, how many lacked a source that measured the claimed thing? Count false support and over-strong interpretation separately from citation formatting errors.

5. Contradictions and corrections. Did it find sources that conflict with a proposed claim? Did it recognize corrected results, retracted work and changes in study reporting? Report both misses and false alerts.

6. Reproducibility and cost. Can an independent group reach the same bounded conclusion using frozen inputs, versioned methods and the same decision rules? Record human verification time, compute cost and failures, not just automated processing speed.

For all six, show the denominator and the evaluation design. For a hypothetical result in which 19 of 20 cases passed, say which 20 were selected and what failed. If a metric cannot be measured, report not evaluated, not “excellent.”

A confident causal leap deserves its own check

Consider a hypothetical review that moves from a drug-level observation to a claim about one receptor. The decision to accept that claim needs evidence that isolates the receptor and the relevant outcome.

A good AI-assisted review must notice when a publication's explanation is stronger than its measured result.

In this hypothetical example, a study might report that a drug was associated with an improved biomarker. That is a result at the drug and biomarker level. It does not automatically prove that one named receptor caused a patient benefit.

A scientific system should be rewarded for refusing that leap when the experimental design cannot isolate it.

This is why our public Not Evaluable Is a Result case matters: a plausible scientific hypothesis was not promoted because the necessary causal evidence was not available in the admitted public data. The case is part of our own four-experiment technical preprint, which is not peer reviewed and does not establish external validation of the workflow.

Independent review must be independent in substance

Asking two AI systems to approve the same polished report may surface useful disagreements. It does not replace independent human scientific judgment, and agreement does not establish truth.

An effective challenge should specify what each reviewer can see, whether the reviewers share model families or training assumptions, what they are asked to falsify, and how disagreements are resolved against the actual sources.

For a serious scientific benchmark, qualified human reference judgments should be established independently. Reviewers should not be shown the AI's preferred answer before they set the reference standard.

The International Committee of Medical Journal Editors (S2) emphasizes that humans remain responsible for the accuracy, originality, attribution and disclosure of AI-assisted scientific writing. We apply those principles to this company article as editorial practice; ICMJE guidance is written for scientific publishing, not as a legal requirement for company blogs.

A practical evaluation card

Before trusting an AI-assisted scientific report for an important decision, request one page with the following fields:

A sales deck with screenshots, testimonials and speed claims is not a substitute for this card.

What we can claim today and what remains untested

Biodius has published its own prospective public-data experiments and made falsification, evidence independence and calibrated uncertainty central to its method. Those experiments show what our team tried and what happened in those particular settings.

They do not establish superiority to experienced human reviewers, clinical validation or reliable performance across unseen disease areas. We should not make those claims without the independent tests described above.

In our view, a meaningful next scientific milestone would be a prospective, independently adjudicated methods evaluation that publishes its errors as clearly as its successes.

That would give a biotech R&D team, a research institute or a grant consortium something they can actually inspect.

Trust should be earned by measured behavior, not by fluent prose.

If your team is evaluating scientific AI or would be interested in an independent, pre-specified methods comparison, learn about Biodius scientific diligence or contact First AI Movers. This is an invitation to explore a prospective evaluation, not a claim that one is already independently certified.

Frequently asked questions

Are the 17% and 63% figures Biodius results?

No. They are task-specific medians reported by an independent living rapid review of heterogeneous studies. They do not measure Biodius performance or establish a universal AI error rate.

Does agreement between two AI reviewers establish validity?

No. Agreement can be a useful prompt for investigation, but the reference facts and interpretation still need independent source checks and qualified human judgment.

Has the proposed methods comparison already been completed?

This article proposes an evaluation design. It does not report a new benchmark, independent certification or clinical validation of Biodius.

Read further

• When an evidence gap is a useful result
• Our public account of the preprint and its limits
• About First AI Movers and its editorial perspective
• Biodius Target Challenge FAQ: Scope, Inputs, Price and Limits

Our source is Costa (2026), Falsification-First AI for Biological Target Assessment: Lessons from Four Prospectively Frozen Experiments, DOI 10.5281/zenodo.23057911 (S3). This preprint is not peer reviewed and has not been externally validated; it establishes no diagnostic, therapeutic, safety or predictive claim.

AI assistance was used for drafting, source checks and editing. Human editorial responsibility remains with First AI Movers.