Four Papers, One Trial: How to Count Independent Evidence

TL;DR: Map reports to underlying studies before judging biological evidence, with a hypothetical trial example and a source checklist.

One clinical trial may generate a registry record, a conference abstract, a journal article and a later subgroup analysis. That is four useful documents, but it is not four independent clinical trials.

Why this matters: a biotech science lead, venture diligence reviewer or research partner needs to know whether several citations describe independent evidence.

This distinction sounds obvious until a scientific team has to make a decision from a large pile of papers. A review may cite an earlier review. A conference result may later appear in a journal. Safety details may appear in a registry while efficacy outcomes appear in a journal. The same group of patients can turn up again in a follow-up report.

When those documents are treated as independent confirmations, the evidence may look more convincing than the underlying studies warrant.

The fix is not to throw away secondary reports. It is to identify the experiment behind each report and decide what new information each document actually contributes.

The document is not the study

A paper is a way of describing research. The study is the research that was done.

For a clinical trial, the study normally has a defined protocol, recruited participants, treatment groups, outcome measurements and an analysis plan. One study may produce several public records and many scientific questions. Those records can be helpful in different ways without adding an independent population.

The Cochrane Handbook (S1) is explicit: systematic reviewers should link reports of the same study, and count the study, rather than each report, as the unit of inclusion.

That matters equally outside formal systematic reviews. A target-diligence report should not call repeated reporting “replication” when no independent experiment has occurred.

A fictional example: four documents, one cohort

Imagine a research program called Study Z. The following example is hypothetical; it does not describe any named medicine or company.

Now imagine an unrelated team repeats the question with newly recruited patients and a genuinely independent study design. That would supply another line of evidence, even if it disagreed with Study Z.

The lesson is not “four papers are worthless.” It is that the four documents can strengthen completeness of reporting, but they cannot each be counted as another independent replication.

The practical source map

The fields below are our editorial checklist, informed by the reporting guidance; they are not a new validated assessment scale.

When comparing biological target evidence, we recommend a map with one row per underlying study. The map should record:

Study identity. A registration or accession number, protocol identity, title, sponsor, design and study dates. Where identifiers differ, the review explains the match rather than guessing.

Sample identity. Who or what was measured? The same recruited people, clinical sites, tissue bank or cell lines may occur in multiple papers. If overlap is unknown, independence is unknown rather than presumed.

Experimental comparison. What actually varied? A drug, dose, genetic variant, receptor, tissue, diet or analysis method? These are not equivalent interventions.

Outcomes and time windows. In a hypothetical reporting mismatch, a safety figure measured over a year should not be silently paired with a weight or biomarker mean measured at week 20. Patients included in the safety population may differ from those included in an efficacy analysis.

Source hierarchy and conflicts. Which report supplies the actual number? If a journal publication and a registry record report different estimates, the review should check whether they use different populations, estimands, follow-up times or corrections. A difference is not automatically an error.

Correction and provenance status. A corrected paper is not identical to the original version. A retraction, amendment or changed registry entry needs a dated source-specific note.

This map makes the evidence checkable. Without it, the same source can circulate through several summaries and acquire the appearance of consensus.

Replication is not simply another impressive result

True replication also asks whether a result survives a meaningful independent test.

Suppose a signal appears in one patient tissue, but an independent dataset sampled a different tissue. A failed match can limit confidence, yet it may not falsify the original context-specific biology. The difference between replication failure and context mismatch must remain visible.

Our own public case study, 0.992 Was Not Replication, describes a striking signal from one dataset that did not reproduce in independent blood T-cell data. Because the discovery cells and test cells came from different tissue contexts, the outcome did not prove the original biological idea false. It showed that the initial signal did not transfer to the tested data.

This was one of our own prospective public-data experiments, described in a technical preprint. The preprint is not peer reviewed and has not been externally validated.

Three ways double counting misleads a team

First, it can inflate confidence. If one trial is cited repeatedly, a reader may see a large bibliography and mistake it for independent confirmation.

Second, it can hide disagreement. A publication may focus on efficacy while the registry holds more detailed adverse-event or exploratory outcome information. Both belong in the record, including genuine differences.

Third, it can confuse biological causality. Two papers describing drugs that affect related pathways do not prove what one particular receptor, gene or cell type caused. If different components changed together, the causal claim still needs a study capable of separating them.

The CONSORT 2025 statement (S2) (DOI 10.1136/bmj-2024-081123) emphasizes transparent reporting of how a trial was designed, conducted and analyzed, including alignment with protocols and registries. Consistent reporting supports critical reading; it is not a substitute for independent replication.

A five-question check before trusting the bibliography

When someone presents a list of studies backing a biological decision, ask:

1. How many independent underlying studies are there, after multiple reports and overlapping samples are joined?
2. Which studies measured the actual patient or biological outcome the decision depends on?
3. What differences in population, intervention, comparator and time point prevent direct comparison?
4. Which conflicts and corrections have been investigated rather than averaged away?
5. Where would one additional independent study have the greatest chance of changing the conclusion?

A strong research report should make these questions answerable from its source appendix rather than require the reader to trust a polished summary.

What this means for scientific diligence

Evidence is not a pile of citations. It is a chain of identifiable observations, analyses, comparisons and limits.

An honest review may find that a biological story has substantial support from independent groups. It may instead find that most public claims trace back to a handful of experiments. Neither outcome can be decided by the length of a reference list alone.

For a consequential decision, show the study map before showing a confidence statement.

At Biodius, this kind of independence check is part of our falsification-first scientific diligence. We distinguish what an assay or cohort observed from what the evidence actually establishes about a target. This method establishes no clinical or predictive validity and does not replace specialist scientific review.

If you are preparing a target or program decision, contact First AI Movers / Biodius with the decision and public evidence you are authorized to share.

Frequently asked questions

Should a review discard secondary reports?

No. They can add methods, outcomes or later observations. Link them to the underlying study and explain which report supplies each result.

Does a later analysis of the same participants count as replication?

It can answer a new question without adding independent participants. Record that contribution and the shared study identity separately.

What if participant overlap cannot be resolved?

Label independence as unknown. Explain the unresolved match and how it limits the conclusion instead of silently treating the reports as separate confirmations.

Read further

• Why a discovery signal did not replicate in the tested context
• Our public account of the preprint and its limits
• About First AI Movers and its editorial perspective
• Biodius Target Challenge FAQ: Scope, Inputs, Price and Limits

Our source is Costa (2026), Falsification-First AI for Biological Target Assessment: Lessons from Four Prospectively Frozen Experiments, DOI 10.5281/zenodo.23057911 (S3). This preprint is not peer reviewed and has not been externally validated; it establishes no diagnostic, therapeutic, safety or predictive claim.

AI assistance was used for drafting, source checks and editing. Human editorial responsibility remains with First AI Movers.