Many, if not most, peer-reviewed biomedical studies cannot be replicated independently. Surprised? Researchers working to develop new drugs that have tried to replicate publications are not.
A recent survey in PLOS One of researchers at MD Anderson, one of the premier Cancer Research institutions in the world, again substantiates the poor reliability of published research that has come up repeatedly over the last few years. Over half of the investigators surveyed said they had been unable to reproduce findings found in the published literature and only a third of these cases were eventually resolved.
The PLOS One survey of MD Anderson researchers also provides some indication of the cause of the problem. Almost a third of the research trainees at the institution felt pressure to generate data in support of their mentors' hypotheses, and 18.6% reported that they had been pressured to publish data they were unsure of.
Actually, concerns about the issue and possible solutions have been raised for several years. However, solutions to the problem have been sparse. Meanwhile, the ubiquity of irreproducible results creates a costly problem for biomedical research.
Lots of time and money is spent by pharmaceutical and diagnostic companies to pursue promising published results that indicate new approaches to identify and treat diseases. When these efforts do not pan out because the original work on which they are based is flawed, this money and time is lost. Meanwhile, other projects based on more solid science that could produce new treatments or diagnostics are delayed or passed over. Ultimately, specious published data hurts all of us because it impedes the research that will develop better cures.
About a year ago, Elizabeth Iorns, CEO of Science Exchange, teamed up with colleagues at PLOS One, Mendeley, and figshare to start the a program that attempts to address this problem head-on by setting up a mechanism for the scientific community to replicate and validate important scientific findings. Recently, I had the chance to ask Dr. Iorns her thoughts about how the project got started and is working out. You can read her comments about Reproducibility Initiative here.