Better follow-up, but unregistered again, and too many endpoints

Yesterday (click here) I commented on a postnatal depression trial run with the parenting website Netmums, in which only 1/3 of participants had been evaluated for the primary endpoint, a depression score, and no health events had been reported at all.

The same authors have just reported another trial (click here), also using Netmums.  This time participants were pre-evaluated by telephone interview and those with a diagnosis of major depression were randomised to i-BA or treatment as usual. Follow-up was higher; by 26 weeks 31/41 (76%) participants in the intervention group and 28/41 (68%) controls remained.

Again the treatment appeared to work, but again the trial was unregistered. This matters because the authors report multiple endpoints, but readers don’t know which was primary.

There were five separate outcome scores (EPDS, Edinburgh Postnatal Depression Scale; GAD-7, Generalized Anxiety Disorder Screener; WASAS, Work and Social Adjustment, Scale; SPS, Social Provision Scale; PBQ, Postnatal Bonding Questionnaire) plus six healthcare utilisation measures, as well as antidepressant use. Twelve endpoints, each measured twice, at 17 and 26 weeks, makes 24 endpoints! For some raw scores were compared, for others change scores, and some analyses were done with, and some without, adjustment for baseline imbalances.

The 25th outcome was potentially the clinically most important; a “reliable and clinically significant improvement in EPDS” at 17 weeks. In the i-BA group 23/37 reported this v. 10/34 among treatment as usual. The odds ratio after adjustment for baseline imbalance appeared to be statistically significant*. But the authors didn’t define “reliable and clinically significant” improvement. And even if they did, was it the predefined primary endpoint?

Researchers funded by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research (CLAHRC) should do better.

Jim Thornton

* The text actually reads: “After adjustment for baseline EPDS the odds ratio for a reliable and clinically significant improvement in the treatment group compared with control was 0.26 (95% CI 0.10–0.71)”. This suggests reduced odds of improvement in the treatment group, i.e. the opposite of the raw data. Presumably they meant “… the odds ratio for not having a reliable and clinically significant improvement …”