Continued violations of key integrity rule fuel evidence distortion in medicine – new study
A new study published today shows that rules on clinical trial registration, a key research integrity safeguard for drugs and medical devices, are still being routinely violated, leaving the door wide open to evidence distortion in medicine.
All trial reports reviewed by the study were published in journals widely read by practicing doctors, and are thus highly likely to influence prescription behaviour and treatment choices.
The findings show that governments around the world are still failing to effectively enforce trial registration rules. Registration of (some or all) clinical trials is a legal requirement in many jurisdictions, including the United States and the European Union.
The authors of the new study examined 486 clinical trial reports published in ten academic journals found that:
47 trials (10%) had not been registered at all
A further 99 trials (20%) had been registered late
101 publications (23%) showed signs of evidence distortion
Unregistered trials were more likely to report positive findings
According to the authors of the study,
“unregistered trials reported favorable findings at a higher rate than trials that had registered. The publication of unregistered trials and trials registered after initial primary outcome ascertainment raises concerns about selective reporting and the integrity of reported outcomes, as these trials are vulnerable to potential changes obscured from public record.
Despite policies to improve registration rates, publication of unregistered trials persists. Our study demonstrates that even the highest-impact journals associated with US professional medical societies publish unregistered trials…”
Until policy makers put into place strong laws, effective monitoring systems and sanctions with real teeth to ensure that all trials are registered and reported, patients like Emma Yasinski will continue to pay a steep price. Click on the image below to read her story.
The study's finding that unregistered trials were significantly more likely to report positive findings illustrates the dangers of failing to enforce prospective trial registration. If scientists fail to specify in advance what they will measure, they can later trawl through huge data sets and cherry pick those results most likely to portray a drug, device or treatment in a positive light, thereby distorting the medical evidence base.
In addition, 101 articles (23%) based on registered trials reported primary outcomes different from those initially specified, a form of scientific malpractice known as ‘outcome switching’. In these cases, scientists did originally pre-specify what would constitute a successful outcome, but then moved the goalposts after the results were in.
The trials in the cohort were conducted in a variety of different countries, and were registered across several different registries. Trials not funded by industry were more likely to not have been registered, presumably reflecting weaker systems and oversight in the public and academic sectors.
All articles were published between 2010 and 2015, years after the International Committee of Medical Journal Editors (ICMJE) had announced that unregistered trials would no longer be accepted for publication. All ten academic journals in the cohort subscribe to ICMJE standards, but nine of them published unregistered trials regardless.
The new study will come as a wake-up call to policy makers and the public. Many people mistakenly assume that following numerous scandals and efforts to solve the problem, the issue of trial registration has long been ‘solved’. Clearly, that is not the case.
The study's findings and conclusions are broadly in line with the wider literature on trial registration and outcome switching. A study published earlier this year found that over 95% of articles rejected by conscientious editors at one journal due to unsatisfactory trial registration subsequently got published by other journals.