Pressure on the FDA to approve an unproven drug – GWC Mag

This article about the pressure on the FDA to green light an unproven drug is by Aaron Charlton, Ph.D., a science blogger and entrepreneur who works for Away Clinic, a travel vaccine provider in Arizona. Aaron’s focus is on improving the quality and transparency of scientific research and is sometimes quoted by the media on matters of scientific integrity.

In the fall of 2021, the U.S. FDA issued a “refuse-to-file” letter to Stealth BioTherapeutics regarding the company’s proposed use of elamipretide to treat Barth syndrome, a rare genetic disorder. The FDA’s action followed a drug trial that failed to meet its primary endpoints a year earlier. Now, the FDA is under public pressure to review and approve this unproven drug.

The pressure campaign

Pressure on the FDA has taken many forms, including a petition with 20,000 signatures, an op-ed by a physician specializing in mitochondrial diseases, and media outreach. The pressure campaign is generally based on the idea that the FDA is simply not evaluating the drug as a priority due to the rareness of the disease. The drug’s efficacy has not been proven, as I explain below.  

It’s not at all surprising that parents would desperately push for anything that might help their child. By all accounts, Barth syndrome is a tragic disease. Due to an error in the genetic code, the cell’s mitochondria fail to function correctly, leading to a whole host of systemic issues.

The heart, muscles, growth, and immune system are particularly negatively affected. It’s also not surprising that healthcare providers are hoping for a miracle cure. It must be painful to see patients suffer so horribly at such a young age from an incurable, untreatable disease. 

But statistics come first

Healthcare providers have to set emotions aside, however, and evaluate evidence about treatments with an unbiased lens. The public relies on physicians to interpret drug trial protocols and outcomes intelligently. Physicians wield enormous power in determining what treatments are available to the public.

Not only are they the primary prescribers, but they are also heavily involved with the execution of drug trials and are the go-to experts the media uses to debate drug-related issues. Given the damage ineffective and unsafe drugs are capable of, this is a tremendous responsibility. 

Randomized clinical trials failed to show an effect

Let’s look at why the FDA may not have approved the unproven drug. The drug trial takes place in two parts. The first part is a randomized control trial (RCT), the gold standard for drug testing. The second part was an unblinded continued use of the drug to test it for safety over a longer period. Due to the rarity of the disease, only a small pool of participants was used for the trial (presumably because that’s all that was available). 

Here is an overview of the study protocol:

Following the randomized clinical trial (RCT) portion of the trial, there was no statistical difference between treatment and control on either primary endpoint:

1. 6-minute walk test distance (-0.8 m, p=0.97)
2. BTHSA-SA total fatigue score (+0.06; p=0.89)

No statistical differences were observed in any of the secondary endpoints, including muscle strength tested with handheld dynamometry (+6.7 newtons; p=0.65).

This indicates a failed drug trial. 

What do we learn from trials with small sample sizes?

The RCT portion of the trial failed to show any effect at all, but in fairness, the 12-person trial was probably underpowered. Finding test subjects that met the study inclusion criteria would have been very difficult. Barth syndrome is an exceedingly rare disease, and only around 10 children are born with the disease in the U.S. each year.

Only very large effects show up in such small studies. A null result is inconclusive because even a very small effect is of interest in this case. We don’t know if the drug works. It may work, which future testing may show. We can conclusively say, however, that the RCT yielded precisely zero evidence that the drug had the desired effect of reducing Barth syndrome symptoms.

Open-label extension

Following the failure of the RCT portion of the trial, ten participants continued into an “open-label,” or unblinded, study, in which they were administered 40 mg of elamipretide daily for 36 more weeks. The goal of this continuation of the original study was explicitly to evaluate the “long-term safety and tolerability” of the drug–not to evaluate efficacy (which was the job of the RCT portion of the study).

Two subjects dropped out during the second phase, leaving a total of eight participants for analysis at the end of the second phase. This second part of the study did show some positive effects on the primary endpoints, but it was not an RCT.

Why not use the open-label extension to evaluate effectiveness?

The problem with the study continuation is that it lacked a placebo and blinding, so it wasn’t a randomized control trial (RCT). RCTs are generally preferred in drug testing because the randomization “reduces bias and provides a rigorous tool to examine cause-effect relationships between an intervention and outcome”.

Without randomization, study participants and administrators know what the point of the study is and they may labor to achieve the study’s ends, even without knowing they are doing it. For example, in this study, participants might exert themselves more at each stage to try to achieve a longer walking distance at each measurement.

Their motivation for doing so might be because (a) they believe the drug is effective and they want to help prove it, or (b) driven by a sense of achievement, they hope to beat their previous attempts at earlier stages. Study administrators may unintentionally provide increasing motivation to participants at each measurement point. Keep in mind that everyone involved wants the drug to work. There are no impartial participants or administrators in most cases. This can bias results. 

The open-label study with 8 participants showed an effect on primary endpoints at 36 weeks. While this is a promising development that perhaps warrants further testing, it is not up to the RCT standard of evidence we have come to expect from drug testing.

Conclusions about the FDA approving an unproven drug

We should hold the FDA accountable when they perform their role poorly, but when we see the process working, we should praise them, not launch media attacks. Undue pressure on the FDA to approve drugs that are not safe and effective will have the regrettable effect of weakening their resolve when it comes time to reject the next dangerous drug. They shouldn’t have to worry about public backlash. 

References