Is ‘silent’ outcome switching in clinical trials research misconduct?
Imagine a researcher runs a clinical trial to see whether a drug lowers mortality among people with cancer after two years of follow-up.
When the results are in, she finds that the drug does not lower mortality after two years of follow-up, but after dredging through the data, she finds a decline in mortality after one year.
Prominent scientific journals are unlikely to be interested in a paper about yet another failed cancer treatment, and the tenure countdown clock is ticking. So she writes up a paper touting the drug’s success after one year, omitting to mention that this outcome was not originally pre-specified, and also omitting the two-year data showing no effect on mortality.
This happens all the time
Outcome switching in clinical trials reports is surprisingly common:
A study of 67 clinical trials published in high-impact medical journals found 357 silently added outcomes. Only nine of the 67 trials had been perfectly reported.
A similar study focused on cancer trials also found widespread, and usually silent, switches in primary endpoints.
Most recently, 41% of German academic clinical trials were found to have switched their primary outcomes; virtually none of the publications disclosed that a switch had occurred.
If it is done for good reasons and – crucially – if the switch is disclosed in the publication, it raises no ethical or scientific concerns. Sometimes it can even lead to improvements in clinical practice.
But in the hypothetical example above, the switch is unlikely to be in the best interests of patients or of science, and is not disclosed. But does it constitute a violation of research integrity?
TranspariMED recently contacted UKRIO, the UK Research Integrity Office, for their take.
UKRIO is an advisory body, not a regulatory organisation, and has no formal legal powers. According to its website, “UKRIO fills gaps between jurisdictions, where no overall regulation might apply.”
TranspariMED asked UKRIO “how 'silent' outcome switching should be categorised. For example, is it a "violation of research integrity" or "unethical behaviour" or "research misconduct" or just a "[Questionable Research Practice]"?
Below is UKRIO’s response, verbatim.
What does UKRIO say?
We are happy for you to make this response public, as we also believe it is important to discuss these issues and have clarity.
Information on the role and remit of UKRIO and its Advisory Service is available on our website, under the heading ‘Guidelines for seeking advice’. Please note that any comments made or views expressed are based solely on the information which you have supplied to UKRIO. UKRIO recognises that this information may be incomplete and/or inaccurate and has made comments in good faith.
1. Changes to research methodology or reporting, such as outcome switching, should be declared openly and transparently, accompanied by sufficient explanation and discussion to provide context for the change.
2. Researchers must also ensure the fulfilment of any regulatory, legislative, governance or other requirements when making changes to methodology or reporting, for example making sure they have sought and received approval from the relevant research ethics committee.
3. Problems arise when changes are not declared openly and transparently, in accordance with good practice standards, whether this is because of a mistake or deliberately.
4. Research is valued by the public and has benefits for society when it has integrity – when it meets standards for good research practice. When research is not honest, rigorous or transparent, or when it is conducted without care and respect or accountability, this harms quality and ethical standards. It damages not only the wider record of research, but also public trust in research and in researchers. Research that lacks integrity can also potentially harm, rather than benefit, society.
5. In general terms, UKRIO would view undisclosed or undeclared (‘silent’) outcome switching as a breach of commonly accepted standards for good research practice, including its Code of Practice for Research.
6. In UKRIO’s view, while occurrences should be assessed on a case-by-case basis, ‘silent’ outcome switching is likely to be in breach of the UKRIO Code’s standards for:
a. Research involving Human Participants, Human Material, or Personal Data - sections 3.6.13 and 3.6.14.
b. Dissemination of Research Outputs - section 3.14.1 "...duty to disseminate research outputs in a manner that reports the research and all the findings of the research accurately and without selection that could be misleading".
c. Dissemination of Research Outputs (sections 3.14.3 and 3.14.12), if the non-disclosure was a result of external pressures.
7. To determine whether an occurrence of ‘silent’ outcome switching was the result of error, questionable research practice or misconduct/fraud would require an assessment on a case-by-case basis using a process such as UKRIO’s Procedure for the Investigation of Misconduct in Research.
8. There is no universal definition of research misconduct, however a common feature of many definitions is that misconduct involves intent to deceive and/or recklessness in the conduct of research. You may find this discussion of the topic by UKRIO useful: What is research misconduct?
9. Research misconduct definitions, including the UK’s (see The Concordat to Support Research Integrity, Commitment 4), commonly include fabrication and falsification (e.g. of data/results). The UK also defines “failure to meet legal, ethical and professional obligations” and “misrepresentation of data, including suppression of relevant results/data or knowingly, recklessly or by gross negligence presenting a flawed interpretation of data” as research misconduct. Please note that UKRIO has adopted the definition of research misconduct from the UK Concordat.
10. When an investigation concludes that ‘silent’ outcome switching resulted from an intent to deceive and/or recklessness in the conduct of research, this could be viewed as falling with the categories of research misconduct set out in (10), above. However, this can only be assessed on a case-by-case basis, using a properly constituted procedure carried out fairly, thoroughly and transparently by an appropriate body (e.g. the relevant research employer or a national research integrity agency, depending on the jurisdiction in question).
11. ‘Silent’ code switching can therefore arise as a result of: honest errors or ‘sloppiness’; questionable research practices or unethical behaviour; or research misconduct. Regardless, it is a breach of good research practice and associated standards. The degree of the breach (fabrication or falsification at the serious end) and the intent are important factors alongside any detriments to knowledge.
Let the debate begin...
We hope that UKRIO’s response will stimulate and inform further debate on where exactly to draw research integrity red lines, and how institutions should respond when study outcomes are ‘silently’ switched.
Note that UKRIO’s remit runs across all research disciplines – so their answer applies to psychology and economics as much as it applies to clinical trials.
TranspariMED would like to thank UKRIO’s team for the time and effort they put into this response.