Is this fraud?

My colleague, Susan Bewley, drew my attention to the Journal of Sex Research last week (click here or Spielmans 2020 Re Analyzing Phase III Bremelanotide Trials for Hypoactive Sexual Desire Disorder in Women). The author, Glen Spielmans, Professor of Psychology at Metropolitan State University in Minnesota, dissects two pivotal phase-3 trials (click here or Kingsberg Obstetrics & Gynecology November 2019), which had led the US Federal Drug Administration (FDA) to license bremelanotide, trade name Vyleesi, to treat women with hypoactive sexual desire disorder (HSDD).

One of the original co-primary outcomes in each trial was switched, many pre-specified outcomes went unreported and new ones appeared, scores were reported as dichotomous rather than as the prespecified means, cut-off values varied for no apparent reason, and other outcomes were claimed as supportive without data. Spielmans also noted that more women (18 or 19% bremelanotide v 3 or 9% placebo) dropped out of the active groups, and questioned whether the various shifts in subsets of sexual behaviour scores, even in the unlikely event they were real, were of clinical significance.

He is correct on all counts. But altered co-primary outcomes in two drug licensing trials? How did that get past the FDA?

Background

Bremelanotide is a polypeptide, related to melanocyte stimulating hormone (MSH). Ever since someone noticed in the 1960s that MSH caused rats to become sexually aroused, chemists have been fiddling with the pharmacology in the hope of creating a female Viagra. In 2016, Palatin Technologies, who own the rights, convinced themselves in a phase-2 dose-finding trial (PT-141-54 in the FDA report) (click here or clayton 2016) that the stuff worked at higher doses. The planned sample size was 100 per group. The pre-specified primary outcome was change in satisfying sexual events (SSE) per month.

	Placebo	Low dose	Medium	High dose
No. randomised	99	100	99	99
Analysed (at least 1 dose & 1 follow-up visit)	91	87	75	74
Dropped out	8	13	24	25
Additional satisfying sexual events per month	0.2	0.6	0.7	0.8

The authors claimed a dose response relationship (bottom row) but were coy about the row above – I’ve extracted it for you. Without knowing what they were getting, three times as many women on the higher doses dropped out. In a double-blind trial this can only mean one thing, side effects. There were many more, mainly nausea, flushing and headache, in the active treatment groups

Such non-random differential drop out pretty much invalidates the results. If women who stayed the course were more stoical, or had a greater desire for their sexual problems to be sorted, it’s hardly surprising to see a small increase in SSEs.

But what do I know? These data were judged sufficiently promising to set up two pivotal efficacy trials for FDA approval. 

The trials

Both had the same title “Study to Evaluate the Efficacy/Safety of Bremelanotide in Premenopausal Women With Hypoactive Sexual Desire Disorder (HSSD)” on Clinicaltrials.gov.

They both studied the allegedly most effective, high dose of bremelanotide, 1.75mg, from the dose-finding study.

They were both reported in the same paper (click here or Kingsberg Obstetrics & Gynecology November 2019). For the FDA analysis (click here or FDA bremelanotide).

Study 1 (labelled BMT-301 in Kingsberg and FDA papers) had a planned sample size of 550 (actual 723) and ran from December 2014 to July 2016, (registry here). Study 2 (labelled BMT-302) had the same planned sample size of 550 (actual 714) and ran from January 2015 to August 2016, (registry here). I could find no explanation for the 30% sample size increases, in either the Kingsberg paper or the FDA report.

Despite their identical eligibility criteria, and almost identical recruitment periods, both trials appear to have been run in the same 91 centres.

The original co-primaries for both trials were:

1.       Change in Female Sexual Function Index–desire domain (FSFI-D). The desire domain is the sum of; “Over the past 4 weeks, how often did you feel sexual desire or interest?” – responses range from 1 “Almost never or never” to 5 “Almost always or always” – and; “Over the past 4 weeks, how would you rate your level (degree) of sexual desire or interest?” – responses range from 1 “Very low or none at all” to 5 “Very high”.

2.       Change in absolute number of sexually satisfying events (SSE) per month.

Of thirteen planned secondary outcomes, the twelfth was this question from the Female Sexual Distress Scale–Desire/Arousal/Orgasm (FSDS-DAO); “are you bothered by low sexual desire?” Responses range from 0 “never” to 4 “always”.

Both trial results would have been negative had the co-primary outcomes remained unchanged. Although absolute numbers of SSEs are not reported in either the paper nor the FDA report, the latter states “neither treatment nor placebo arm showed any improvement” (p 126).

This looks suspicious. SSEs are more meaningful than “bothered by low sexual desire” which is little more than a different aspect of the other co-primary.

But on 27 October 2014 (click here) the FDA had held a workshop with patients about Female Sexual Dysfunction.

The company claimed that workshop participants had stated that SSEs were a poor efficacy outcome. Some quotations e.g. “Several [participants] stressed the importance of feeling desire regardless of whether it is accompanied by a satisfying sexual event” (p6), could be interpreted this way, although others e.g. “Participants largely appeared to believe that having satisfying sexual events was important to them” (p13) suggest the opposite. Even on its own terms this is hardly a convincing argument.

But the workshop had been captured by pharma. Here’s what independent experts, Leonore Tiefer, Ellen Laan and Rosemary Basson wrote about it at the time (click here or Tiefer),

Participants had their expenses paid by [a group funded by] pharmaceutical companies, […] had met the morning of the first day in their hotel to hear presentations and prepare their talking points.

They had each received a green shawl, identifying them with the ‘’even the score’’ campaign that accused the FDA of sexism in handling [female sexual disorder] drug applications [and] arrived at and departed together by chartered bus.

It looks like the FDA was pressured into agreeing the switch, to avoid accusations of sexism in the way they treated applications for female and male sexual treatments.

And the timing of the switch? The alteration was requested on 10 October 2016. Study BMT-301 had been completed on July 26 that year, and study BMT-302 on Aug 4th.

Let’s summarise. More than two months after the studies had been completed, on the basis of a dodgy, pharma-captured workshop conducted two years earlier, the trial sponsors persuaded the FDA to allow the co-primary endpoints to be switched. They were changed from a meaningful predefined outcome justified by the pilot dose-finding study, the number of satisfying sexual events, to the twelfth of thirteen secondary outcomes. And guess what? Bremelanotide, which did not alter the number of satisfying sexual events, magically improved this suddenly important secondary outcome.

Spielmans doesn’t say it, but I do. This must be a data-driven outcome switch. If so, that’s fraud. 

Jim Thornton

Update 10 April 2021

Yesterday Dr Kingsberg and her colleagues replied to Dr Spielmans’ critique (click here or paper), and Dr Spielmans replied to her reply (click here or paper). Despite much bluster, in my opinion, she failed to deal with any of his substantive points. But judge for yourself.

With regard to whether the co-primary outcome switch was data driven, Kingsberg writes;

He also alludes to “data peeking” in his introduction and that the FDA allowed the “sponsor’s request for satisfying sexual events (SSEs) to move from the co-primary to the key secondary outcome …. a year after the trials had begun.” What Spielmans omitted is that the FDA published a guidance document (2016) for designing clinical trials in which SSEs were no longer required to be a primary endpoint for HSDD treatment trials. Instead, trials could now include measures reflecting the hallmark criteria of the condition: loss of (i.e., deficiency or absence of) sexual desire (i.e., FSFI-D) and distress about lack of desire (i.e., FSDS-DAO #13). The approval from the FDA to change the primary endpoint, after discussion with the FDA review division, came prior to the data lock [my emphasis] in these well-conducted, randomized, double-blind, multicenter placebo-controlled trials, with pre-established statistical analysis plans. 

Note the non-denial choice of words. No-one doubts that the outcome switch came “prior to the data lock”. The issue is whether the sponsor had seen the data prior to the outcome switch request.