Two/three arm? 300/1250 participants?

This randomised trial of different methods of neonatal male circumcision in Botswana (click here, or Plank Botswana circ trial), has just been published online in J Acquir Immuno Defic Syndrome. The paper version will appear on April 15th.

The trial was led by US researchers, sponsored by Brigham and Women’s Hospital in Boston Massachusetts, an affiliate of Harvard Medical School, and funded by a grant from the United States National Institutes of Health. This is non-therapeutic research on a vulnerable group – participants were healthy newborn boys born in three government hospitals in Botswana. You’d expect such an ethically sensitive trial, performed by researchers from some of the world’s leading academic institutions, to be done correctly.

First inspection looks promising. The trial was registered in Sept 3rd 2009. This is four months after it started, which is not ideal, but over a year before it ended, which should be early enough to prevent the researchers altering the sample size or primary endpoint to get the result they want. Unfortunately there are serious inconsistencies between registration and publication.  I’ll describe two, the sample size and design, and the primary endpoint.

Sample size and design

The published paper describes a two-arm randomised trial including 300 infants, comparing the Mogen clamp method (n=153) with the Plastibell device (n=147). There is no mention of any alteration in design or sample size.

However the trial registration on ClinicalTrials.gov (here) describes a three arm trial with the AccuCirc device added as a third arm, making a total sample size of 450! Various older versions of the trial registration give a “total anticipated enrolment” of either 1000 or 1250. On the latest version “anticipated enrolment” of 1250 has been altered to “achieved enrolment” of 1020.

What is going on?  It’s bad enough to alter your sample size between registration and publication without giving a reason. To change a three-arm trial into two-arms makes a mockery of trial registration. Was the registration wrong, or are the authors failing to report the third AccuCirc arm?

Primary outcome

In the paper the primary outcome is ambiguous – undefined in the abstract, and defined only as “adverse events and parental satisfaction” in the methods/outcomes section. The statistical analysis section states “this study was designed to detect a 20% difference in parental satisfaction between the 2 techniques”.  

In the trial registration document the primary outcome is clear. In all versions it is “bleeding within six weeks”.

This is not reported as such in the paper, but since most bleeding occurs  early, the outcome “bleeding”[with no time period given] is probably the same.  Five cases occurred in the Mogen group and zero in the Plastibell.  The authors did a chi-square test and estimated P=0.03. i.e. nominally a positive result. This finding makes it into the abstract.

Unfortunately, there are other statistical problems. In the statistical methods section of the paper the authors said they would use Fisher’s exact test for dichotomous outcomes, which is indeed preferable, and when I redid the test that way, the P value for “bleeding” became 0.06 i.e. not quite statistically significant. Maybe it doesn’t matter, because the abstract conclusion is the opposite of the bleeding data – Mogen is said to be safer because there were two cases of migration of the Plastibell.

The details of the satisfaction questionnaire are not described. and the results are presented only as summary percentages “highly or completely satisfied” at 6 weeks (95% Mogen, 96% Plastibell; P = 0.5) and at 4 months (95% Mogen, 99% Plastibell; P = 0.04).  Despite this nominal statistical significance at four months, and the earlier statement in the paper that maternal satisfaction was a primary outcome, the authors made nothing of this and did not even mention it in the abstract.

I could go on. Read the paper. For now a letter to the editor of J Acquir Immuno Defic Syndrome may bring clarification. I will report back.

But this is non-therapeutic research in newborn children who cannot consent. It is the sort of research that should be only done to the highest possible standards and monitored ethically more closely than any other.  Tomorrow (click here) I’ll take a closer look at the complications these babies suffered, and the trial’s ethical monitoring.

Jim Thornton