Replicability in Scientific Studies

A key component of any experiment or study is the ability to be replicated. Since only one result is not significant, scientists will redo their own other’s experiments in order to make sure the results are consistent. With published work, it is important to ensure that the results being presented are accurate. Scientific work affects every area of our lives, such as medicine or technology. It is of the utmost importance that the different findings in science are true. If not, it could lead to harmful effects on society as well as a reduction in the credibility of science. A drug that does not actually work in the way it is supposed to to could have adverse effects on the body. Technology malfunctioning could range from slight annoyance to a fatal catastrophe.

In recent years, the scientific community has raised alarms about a replicability crisis. In 2015, PLoS Biology ran a study that found that $28 billion a year is spent on unreproducible research (Wheeling 2017). Unreproducible research is not reliable. How does one ensure the results are correct? A survey run by Nature in 2015 questioned a little over 1,500 scientists on their perspective of the replicability crisis. 90% of scientists questioned said there was at least a slight crisis. Previous studies into reproducible results of each field found that 40% of psychology papers and only 10% of cancer biology papers are reproducible (Nature 2016). Even with these dismal numbers, many scientists trust the published papers in their field. The study also found that even though the majority of researchers had failed to replicate an experiment, less than 20% of them had been contacted in regards to reproducing their own work (Nature 2016). The silent acceptance of unreproducible work is dangerous.

There is also difficulty in encouraging scientists to replicate another’s experiment, instead of going forward with their own. Papers that agree with previous research have much less appeal than papers that promise a new discovery. It also may be difficult to publish results that go against a widely accepted paper in the scientific community. So much of funding is based on publishing and discoveries, which is what allows the scientist to continue to conduct research (Sheldrake). In the Nature survey, 37% of scientists were able to publish successful or unsuccessful reproduction papers, while 22% were not (Nature 2016). While some papers are published, reproduction attempts should be a norm in scientific communities, even when they do not seemingly possess a novel discovery, or potential for one. The competitive system of academic research also encourages positive results. So, experiments may be rushed in order to be published.

In biomedicine, there is an increased desire for reproducibility, as that will lead to a more effective clinical trial. Reproducibility ensures the results are consistent, which increases the accuracy of the results and leads to a better chance of the drug working in a human. In 2004, the California Institute for Regenerative Medicine was formed. They created the Accelerated Therapies Public Private Partnership award, which gives funds for private industry to help streamline translational research: laboratory developments that are directly applied to medicine (Wheeling 2017). Since reproduction is key, the monetary benefit can be used to help encourage researchers to spend money on replicability studies, in order to see if a drug or treatment has a greater chance of being effective.

Some scientists are skeptical of the crisis. In looking at the total amount of papers, the frequency of questionable research is very low (Fanelli 2018). The unreproducible studies are minor, not evenly distributed across the fields, and not growing. Fanelli argues that problematic papers do not happen often, for example, the FDA found only 4% of clinical trials between 1977 and 1988 to be problematic enough to investigate. Analyses done on the papers suggested a false-discovery rate of 14%, while showing that the majority of published studies are accurate (Fanelli 2018). Several arguments arise that science is just as reliable as before, and that the declaration of a crisis will lead to new scientists leaving the field. Regardless, there is an agreement that science does need to be transparent and reproducible.

There is no simple solution to this problem. With or without a crisis, experiments should still be tested for reproducibility. Scientific work is important, and so is funding for this work. Perhaps if there was a government fund made for replicating experiments, an incentive could increase for labs to replicate previously published experiments. Potential next steps may include wide-spread acknowledgement of this issue, from scientists at different levels in their careers. It is also common for labs to reproduce their own experiments, either by the same scientist or another. Again, there are considerable costs attached, which is a major barrier. While there is a long-term investment in consistent results, many labs are on a time crunch to publish work. Reproductions of previous experiments need to be encouraged and supported by the scientific community in order to ensure scientific progress.

Works Cited

Baker, Monya. 2016. 1,500 scientists lift the lid on reproducibility: Survey sheds light on the ‘crisis’ rocking research. Nature, 55 (7604): 452-4. doi:10.1038/533452a

Fanelli, Daniele. 2018. Opinion: Is science really facing a reproducibility crisis, and do we need it to? PNAS. 115 (11): 2628-31. doi.org/10.1073/pnas.1708272114

Wheeling, Kate. 2017. Big Pharma reveals a biomedical replication crisis. Pacific Standard. Web. Retrieved from: https://psmag.com/news/big-pharma-reveals-a-biomedical-replication-crisis