Key clinical trial data often inconsistent across registries, study finds
A study comparing key data for 197 clinical trials that were registered across multiple trial registries found widespread inconsistencies and data gaps, raising concerns about the quality and reliability of the global medical research information network.
Background
In theory, the global network of trial registries provides an overview of who is researching what treatments for which diseases. This allows research funders to identify and address gaps in research efforts, and enables scientists to avoid needlessly duplicating trials.
For example, if there are two promising drugs for a certain type of cancer, and several trials are already investigating one of these drugs, new research efforts should be focused on the second drug so that all promising treatments get explored.
An additional benefit of trial registries is that researchers have to specify in advance exactly what they will measure, which removes the temptation to later on move the goalposts to make a trial look more ‘successful’ than it actually was.
Inconsistent outcome measures
The team looked at 197 trials that were registered separately in two or three trial registries. They found that the same trial was often described differently in different registries.
Notably, 13% of primary outcome measures were inconsistent across different trial registries. A further 9% of primary outcome measures were described in insufficient detail, so it was unclear whether they were identical or not.
Other detected inconsistencies included information on who was running the trial, who funded the trial, the status of the trial (e.g. ongoing or completed), and the number of patients who participated in the trial. Also, the results of some trials had been made available on one registry, but not on another.
Concerns about data reliability
The study team concluded that:
“[F]or a substantial proportion of registered RCTs, information about key trial characteristics was inconsistent across trial registries, raising concerns about the reliability of clinical trial registries. Further investigation and harmonization efforts across clinical trial registries appear to be necessary to increase their usefulness.”
“Interventions that might increase the reliability include uploading of study protocols (entire protocols or key parts) or linking trial registries through platforms (eg, WHO International Clinical Trials Registry Platform) and implementing automated systems that detect inconsistencies.”
Why are trial registries not fixing this?
The team also interviewed representatives of several trial registries:
“The representatives reported that there were ongoing efforts to harmonize clinical trial registries, referring to the implemented standardized WHO Trial Registration Data Set and to regular meetings organized by the WHO, as well as bilateral meetings among trial registry representatives… [D]ifferent legislation requirements were mentioned as the main hurdle to achieve better harmonization among trial registries.”
“All representatives assumed that lack of regular updates to all trial registries was the main reason for the inconsistencies among trial registries… With regard to some initiatives that might have improved the situation, most representatives mentioned that regular reminders were implemented not only to add study results but also to update trial characteristics.”
One registry representative told the team that:
“We hear that primary outcomes aren’t even well specified in the primary protocol document itself. So it’s really difficult to have a well-defined primary outcome on the registry when you are working with some material that is suboptimal.”
The study cohort of 197 trials is based on research ethics committee approval documentation from four countries (Switzerland, UK, Canada, and Germany) to which the team had gained access. All trials were approved in 2012. Registries were searched in 2019.
All of the 197 trials included were registered on two or three different registries. In total, seven different WHO-linked trial registries were used.
According to WHO best practices, each clinical trial should ideally only be registered on one single registry. However, in some cases, national laws or funder policies make double registrations unavoidable. In other cases, ill-informed investigators sometimes register trials more than once.
In recent years, some major institutions have adopted policies mandating investigators to register trials on one registry only – unless absolutely unavoidable – to reduce the effort required to keep registry data up to date.