Randomized controlled trials are the ‘gold standard’ of research — but a difficult fit for trans care

In order to provide gender-affirming care, clinicians must sometimes jump through unusual hoops.

In Australia, for example, the government won’t subsidize testosterone treatment for a trans man without a peculiar bureaucratic maneuver; a doctor must say the patient has a testicular disorder. The disorder: Trans men, being trans, are born without testes.

Clinician and researcher Brendan Nolan has advocated for a billing code that would allow government subsidization of testosterone directly for the purpose of gender-affirming care. “And we were told we needed a randomized controlled trial before they would even consider that,” he told STAT.

Politicians in the United States and other critics of gender-affirming care have similarly pushed for more highly-valued research like randomized controlled trials (RCTs) to justify the treatments. As the name suggests, these studies randomly assign participants to either a group that receives the treatment being tested or a control group that doesn’t, and they are considered the gold standard of medical research. But clinicians and researchers say it’s time to rethink the insistence on RCTs, which they say are impractical and almost impossible to perform ethically when looking at the benefits of gender-affirming care like hormones and puberty blockers.

Nolan, a University of Melbourne researcher, actually devised a way to conduct what he describes as a “world-first” randomized trial of gender-affirming hormones. The results of the small study, published last week, showed that testosterone therapy had significant mental health benefits for trans masculine adults. But the difficulty of carrying out the research — Nolan readily acknowledges the study has limitations due to ethical and practical considerations — shows just how challenging these studies are.

“[I] have to commend the team for figuring out the one ethical way to conduct an RCT of gender-affirming care,” Ayden Scheim, an assistant professor of epidemiology at Drexel University’s Dornsife School of Public Health, wrote in an email to STAT. “Not that the naysayers won’t just move the goalposts.”

The people calling for RCTs in order to provide gender-affirming care “are simply not serious,” he added. “Not serious in terms of their understanding of basic principles of epidemiology and public health research.”

The practical and ethical limitations of RCTs

Efforts to limit or ban gender-affirming care often hinge on the argument that the existing evidence — made up mainly of observational studies — is insufficient. In U.S. federal court cases where judges are considering bans on gender-affirming care for youth, controversial medical leaders with financial ties to anti-trans institutions have called such care “experimental” and said they believe there’s insufficient evidence to treat youth.

But there are countless medical treatments that haven’t been tested with “high quality” evidence like RCTs — for reasons such as difficulty enrolling patients, the cost or time it would take to perform such a study, or ethical considerations. In such cases, observational data can be good enough, when there are multiple studies involving many patients in many places that produce consistent results.

Surgeries — with a few exceptions — are usually developed without randomized trials, as it’s clearly difficult to convince people in a control group that they’ve had a major surgery when they haven’t.

While some psychotherapies, including cognitive behavioral therapy, have been tested in RCTs, it can be hard for researchers studying mental health treatments to meet the design requirements of RCTs for a range of reasons; in studies measuring the efficacy of psychotherapies, for example, patients and therapists may already be familiar with different approaches, and are therefore less likely to be blind to the type of treatment being studied.

“There are many treatments that had already been in use for many decades before we decided as a field that RCTs are the gold standard, and we didn’t decide that we were going to stop providing all the treatments until we conducted them all as RCTs,” Scheim said.

When concerns do arise about widely used treatments in other areas of medicine, experts don’t typically suggest withholding those treatments. For example, a small 2010 trial suggested that giving supplemental testosterone to cis men with low levels of the hormone could heighten the risk of heart attacks. A much larger follow-up study this year allayed those concerns — but in the meantime, physicians had continued prescribing the medication as needed without legislative interference.

In a paper published earlier this year in the International Journal of Transgender Health, researchers wrote that RCTs are “methodologically inappropriate” for research focused on the mental health benefits of gender-affirming care for youth, noting many logistical barriers. Hormone treatments, for example, lead to visible physical changes, making it impossible to blindly sort patients into treatment and control groups. The impossibility of hiding who is and isn’t receiving treatment could lead patients assigned to the control group to opt out of research for fear of not receiving hormones, as well as potentially bias any self-reported results.

Such concerns arose in a 2001 study in which researchers attempted an RCT looking at the effects of puberty blockers and growth hormones in cisgender, adopted children who both experienced precocious puberty and were predicted to have a short final height. The team was forced to switch to an observational study because parents of children in the control group refused to participate, wanting to ensure their kids received treatment.

Many of the calls for more research on gender-affirming care particularly focus on the importance of more evidence on puberty blockers. But the 2001 study “really underscores this unfair bias,” said Diana Tordoff, an epidemiologist currently working on the national PRIDE Study at Stanford University School of Medicine and a co-author of the IJTH paper. “We don’t expect cisgender youth to have to undergo this [an RCT]. … But it is being argued that this is expected for gender-affirming care.”

Observational studies are the “bread and butter” of epidemiology, Tordoff said. She and co-author Florence Ashley, a bioethicist and law professor at the University of Alberta, believe that a system of overlapping observational studies can provide the needed rigorous evidence to prove the efficacy and safety of care like hormones and puberty blockers.

And there are already quite a number of observational studies finding mental health benefits of gender-affirming care for both trans youth and adults. The existence of this evidence raises another important ethical concern for RCTs.

The principle of clinical equipoise holds that, when starting a trial, researchers must be uncertain about whether the control group or the treatment group is most likely to benefit. If they can make an assumption, there is an obligation to provide all patients with the best care. At this point, there is so much observational evidence on the benefits of gender-affirming care that most researchers won’t withhold treatment in order to have a control group, especially for longer-term studies.

These types of issues are common for researchers in any field considering RCTs. But critics tend to hold transgender health to higher standards than other, similar areas of medicine, experts say.

“Maybe the larger question is, would anything be enough?” said Zack Marshall, an associate professor of community health sciences at the University of Calgary’s Cumming School of Medicine. “For certain audiences, governments, leaders, it doesn’t matter what the research says. It’s almost irrelevant.”

In pursuit of bigger questions

Still, some researchers haven’t given up on reaching the gold standard. Nolan and his team managed to run a small, short-term randomized trial comparing the effects of immediate testosterone therapy to waiting three months on gender dysphoria, depression, and suicidality. They believe it is the first randomized trial of gender-affirming care that included a control group that received no treatment.

Withholding treatment for three months was ethical because that’s the average wait time for patients to be seen at their clinic, the researchers said.

Nolan said he sees the benefits of hormones “every single day” in his clinic, and launched the study because “there’s obviously a lot of very disappointing things that are being said and a lot of attacks that are actually not based on any evidence.”

The trial of 64 transgender and gender-diverse adults found decreased gender dysphoria, depression, and suicidality in the group treated with testosterone. Notably, half of suicidal patients in the treated group (11 people) reported these feelings resolved during the study compared with just one (5%) in the control group. The results were first presented at the Endocrine Society meeting in June and were published this month in JAMA Network Open.

The trial had important shortcomings. It was not a blinded study, so participants knew whether they were receiving treatment or not. Nolan said it wouldn’t have been ethical to give somebody a placebo, though he recognizes this is a limitation to the study. Still, they found that setting a specific, short time period for withholding treatment helped to reduce the chances that participants in the control group would drop out of the study due to too long of a wait.

There are other issues that researchers would like to dedicate their time to, such as questions about personalizing gender-affirming care to fit the physical changes individual trans people want to see.

“There’s a lot of stuff that can be done, like comparative effectiveness studies and all sorts of RCT-based studies, around producing the exact physiological changes that people want,” said Ashley.

But for now, some researchers feel that they’re stuck playing defense as the basic value of gender-affirming care continues to be questioned.

“Those clinics in many states across the country are literally fighting for their survival because of these attacks,” Scheim said. “They’re not well positioned to be able to do research that would be really helpful, but isn’t as urgent as just keeping the doors open.”