The suggestion was put forward in a paper on the economic impact of irreproducible research this week in which authors Leonard Freedman, Iain Cockburn and Timothy Simcoe argued that common errors mean half of all preclinical trials are impossible to repeat.
The authors looked errors in study design, problems with reagents, materials and lab protocols or mistakes during data analysis and concluded that 53.3% of all preclinical trials conducted cannot be repeated.
If the finding is correct – the estimates are based on a small number of papers where errors were detailed – it would mean around $28bn (€24.8bn) is spent on irreproducible preclinical studies year, based on a total annual spend of $56bn reported in 2013 AAAS research.
They authors also suggested the inability to reproduce research may mean “lifesaving therapies are being delayed" explaining that normal drug industry practice is to repeat academic research before initiating development in a process that can take two years and cost as much as $2m.
“While industry will continue to replicate external studies for their own drug discovery process, a substantially improved preclinical reproducibility rate would derisk or result in an increased hit rate on such investments, both increasing the productivity of life science research and improving the speed and efficiency of the therapeutic drug development processes.”
Solutions
The reproducibility of preclinical research is also focus for the US National Institutes of Health (NIH), which recently introduced reporting guidelines for journals and scientists.
Freedman and his team welcomed such efforts as positive, but suggested “compliance levels and their impact to improve reproducibility have been disappointing” adding that the scale of the challenge means “there is no single magic bullet solution to the problem.”
In clinical research, initiatives like CONSORT have improved reproducibility. However, according to the authors, such measures cannot simply be applied to preclinical studies without a dramatic increase in the cost and time.
STR use low
The researchers also suggested some trials are irreproducible because simple biochemical tools are not being employed effectively, citing the short tandem repeats (STR) that can be used to confirm the identity of cell lines as an example.
“Despite the availability of STR analysis as an accepted standard to authenticate cell lines… only one-third of labs typically test their cell lines for identity. For an NIH-funded academic researcher receiving an average US$450,000, four-year grant, purchasing cell lines from a reputable vendor and then authenticating annually will only cost about US$1,000 or 0.2% of the award."
They estimate the US NIH spends about $3.7bn a year on research involving cell cultures or cell lines and suggest that using STR to reduce the proportion of these studies that are irreproducible as a result of cell line misidentification would have significant economic benefits.