Disappointed!

In 2013, tumor biologist Elizabeth Iorns initiated an ambitious project-Reproducibility Project (Cancer Biology) in the field of cancer biology, which aims to verify 53 high-impact cancer biology articles from 2010 to 2012 The reproducibility of related research papers, and the repeated results will be published in the eLife journal
.

Now, 8 years have passed.
Due to funding problems and lack of transparency in data, reagents and operating methods in some papers, only 23 papers have been replicated in the verification test
.

How repetitive are these papers? The results are disappointing.
Of the 23 high-impact cancer research papers, less than half of the test results can be repeated
.

In fact, this project was difficult from the beginning because many papers lacked the introduction of experimental details.
When the staff contacted the authors of the papers, only 41% of the paper authors provided active assistance, and about one-third of the authors directly Reject or not respond
.

In the end, the project reduced the originally planned 53 papers (including 193 key experiments) to 23 papers (including 50 key experiments), and completed all the key experiments of 18 of the 23 papers, and the rest Part of the experiment in 5 papers
.

Project team used five criteria to evaluate the reproducibility of these papers, including the results are the same direction and have the original papers statistical significance, if repetitive effect size at 95% confidence interval, etc.
can be
.

Among the 23 papers, only 5 of the key experiment results can be repeated (accounting for 21.
7%).
Among 112 experiment reports, only 46% of the reproducible experiments can meet 3 of the 5 reproducible standards and Above
.

As can be seen from the statistical results in the figure below, the repetitive results of 8 papers are positive (5 of them are completely reproducible, and 3 papers only did some of the experiments due to lack of experimental details); the repetitive results of 7 papers are good Mixed; 5 papers cannot be repeated at all; 3 papers cannot be judged
.

These research papers that are completely unrepeatable or partially unrepeatable have tens of thousands of citations, which obviously has a huge impact on the entire cancer biology research
.

Source: Science

These 23 papers include: 5 Cell, 9 Nature, 2 Science, 1 Cancer Cell, 1 Nature Cell Biology, 2 Nature Medicine, 1 Science Translational Medicine, 1 Genome Research, 1 PNAS
.

It should be pointed out that the experiments that cannot be repeated in this project do not necessarily mean that they cannot be repeated.
For example, the papers that cannot be repeated in this project are shown to be repeatable in the papers of other research teams
.

The project leader Elizabeth Iorns said that funders of scientific research projects and academic journals should strengthen the requirements for authors of papers to share experimental methods and materials
.