Too good to be true?

Remarkable how often clinical trials report equal, round numbers in each group. Some hit exactly their planned sample size as well. Two groups in last month’s NEJM even managed it with a three arm trial.

Both trials compared bariatric surgery (gastric sleeve or bypass in Cleveland, biliopancreatic diversion or gastric bypass in Rome), with diet for obese diabetics.  Each trial had an agreed sample size and primary endpoint. Click here and here for the trial registrations and here and here for the published papers. The NEJM is behind a paywall.

The Cleveland trial planned to recruit 150 participants to three equal groups and, after screening 218, recruited exactly that, 50:50:50. No CONSORT flow diagram was provided.

The Rome trial planned to recruit 60 participants to three equal arms, assessed 72 for eligibility and again achieved exactly 20:20:20. The CONSORT flow diagram shows 8 had declined, 2 were ineligible and 2 lived too far away.

Both trials had other problems. Cleveland’s follow-up was lower among the medical (41/50) than surgical patients (99/100), with only a partial explanation hinted at in the footnote to Table 4 “seven patients in the medical-therapy group withdrew immediately after randomisation”. Rome’s registered primary outcome was partial or total remission of diabetes at two years, but the paper only reported total remission rates.

These latter problems must have introduced some bias, but what are readers to make of the improbability of the achieved group sizes?

Even ignoring the difficulty of stopping a trial exactly on the button, randomising in equally balanced blocks of say, 3, 6, or 9 would not have achieved the observed result because neither 50 nor 20 is divisible by three. Balanced blocks or 50 and 20 would have done the trick but they surely didn’t programme a computer to do that! Nor did they shuffle 150, or 60 cards. Cleveland used “a block randomisation method with a 1:1:1 ratio”.  In Rome “patients were assigned […] by simple randomisation in a 1:1:1 ratio with the use of a computerised system for generating random numbers.”

I’m not suggesting cheating, but it is strange.  Is it possible a naïve trials clerk cleaned up the data?

Jim Thornton

Here are the full references.

Cleveland – Schauer PR et al. Bariatric Surgery versus Intensive Medical Therapy in Obese Patients with Diabetes. N Engl J Med 2012; 366:1567-1576

Rome – Mingrone G et al. Bariatric Surgery versus Conventional Medical Therapy for Type 2 DiabetesN Engl J Med 2012; 366:1577-1585

Funding.

Cleveland – Ethicon and National Institutes for Health i.e. mixed private and public.

Rome – The Catholic Universty of Rome. Not clear if public or charitable.