The first results of a high-profile effort to replicate influential papers in cancer biology are roiling the biomedical community. Of the five studies the project has tackled so far, some involving experimental treatments already in clinical trials, only two could be repeated; one could not, and technical problems stymied the remaining two replication efforts.
Some scientists say these early findings from the Reproducibility Project: Cancer Biology, which appear tomorrow in eLife, bolster concerns that too many basic biomedical studies don’t hold up in other labs. “The composite picture is, there is a reproducibility problem,” says epidemiologist John Ioannidis of Stanford University in Palo Alto, California, an adviser to the project whose attention-getting analyses have argued that biomedical research suffers from systemic flaws.
But others say the results simply show that good studies can be difficult to precisely reproduce, because biological systems are so variable. “People make these flippant comments that science is not reproducible. These first five papers show there are layers of complexity here that make it hard to say that,” says Charles Sawyers, an eLife editor and cancer biologist at Memorial Sloan Kettering Cancer Center in New York City.