Bury it quick

We generally limit our reports of trial registry troubles to the major journals, BMJ, Lancet, NEJM, JAMA. For lesser journals we’d need a team of hundreds!

But some cases are irresistible. We learned of this trial (click here) from retractionwatch.com. They had noted a retraction where the alleged reason was that an analysis by intent to treat had been muddled with a per protocol one. RW commended the authors for their honesty.

The trial purported to test the effect of being invited to look at an educational website prior to genetic counselling about breast cancer risks. It was registered here. The sample size (100 per group) and the primary outcome “Counselees’ participation, i.e. content and amount of questions asked and information received during the visit” were pre-specified. Apart from those being three primary outcomes, so far so good.

But none of the three primary outcomes appeared in the paper. Ten secondary outcomes (cleverly numbered 1-9!) were listed on the trial registration site. Of these four were reported correctly in the paper (one twice, same values!), one which was planned as a change score was reported as an actual score, and five were not reported at all.

The authors also found space to report another 28 outcomes. In total eight knowledge scores (table 3), ten “fulfilment of needs” scores (table 4), six scores measuring “risk perception, anxiety and perceived personal control” (table 5), and eight variations on “topics discussed and recalled” (table 6). Tests of statistical significance were performed on all 32 reported outcomes, of which five were nominally “significant” at the 5% level. Failing to report any of your three primary outcomes, correctly reporting only 4/10 of your secondary outcomes and then bunging in 28 other outcomes, must be some sort of record.

Am I being cruel?  Does it matter? After all no-one dies from looking at a website. This sort of pseudo-science pours out of psychology departments all over the world. Buried in Breast Cancer Research, a vanity publisher, sorry, an Open Access journal, no-one would have taken any notice if the authors had not retracted it.  But the Dutch public via the Dutch Cancer Society paid for it. They should ask questions.

Jim Thornton