Skip to content
Advertisements

Extremely preterm baby dies from natural causes

June 19, 2018

Cord prolapse during breech birth

But the head became detached after death, or as The Sun put it last week: “Doctor decapitated baby in botched birth” (click here), and the story flew round the world (click here). The obstetrician was criticised by a public enquiry, but found fit to continue practice. The mob called for blood, and even level-headed lawyers raised their eyebrows.

              

What happened? I only know what was in the papers.

The baby was 25 weeks, i.e. 15 weeks early. Average weight at this gestation would be about 600 grams (1.5 lbs). Not so long ago, this would have been classed as a miscarriage, albeit a late one, and the mother would never have gone near a maternity ward. But over the last couple of generations, advances in neonatal care have pushed the limits of viability ever lower. In the best centres survival rates at 25 weeks are now about 60%, although about half the survivors will have some form of disability (click here). In this pregnancy the membranes had ruptured and the heart beat was slowing due to cord prolapse, so this baby’s chances were probably somewhat worse. 

Nevertheless the tribunal criticised Dr Laxman’s decision not to do a Caesarean. In fact they went further and said it “was negligent and fell below the standards ordinarily to be expected”. Not surprisingly the lawyers raised their eyebrows.

But unless the tribunal members had additional unreported information, this is an odd ruling. Caesarean is not mandatory for preterm breech births. The National Institute for Clinical Excellence (NICE) considered the matter at length (click here) and, even after 26 weeks, limited their recommendation to (my italics) “consider Caesarean […] between 26+0 and 36+6 weeks of pregnancy with breech presentation.” Below 26 weeks they decided not to make any recommendation.

NICE’s reasoning was sound. At term Caesarean is a bit safer for a breech baby. But preterm is less clear. Caesarean might be less traumatic, but it also brings forward the birth; preterm labour is difficult to diagnose, and may stop for days or even weeks. Unless you are certain the baby will deliver rapidly, the risk of Caesarean increasing prematurity often trumps the hypothetical benefit of reduced birth trauma. The uncertainties are such that the NHS has recently commissioned research to see if a randomised trial is feasible.

But what about after the cord prolapse? The baby will certainly die if not delivered within a few minutes.  But it’s not obvious that Casearean even then is the best option. For a very preterm baby in advanced labour, Caesarean is also traumatic, and the baby may deliver quickly vaginally. I know of two cases (not in my hospital!) where doctors did an emergency preterm Caesarean and found an empty uterus – the baby had delivered by the normal route under the drapes while the surgeon cleaned and incised the skin! 

NICE also considered whether at these limits of viability it is worth monitoring the fetal heart at all. Slow heart rate detects cord prolapse but may also push obstetricians into unwise Caesarean against their better judgment. It’s difficult to refuse a maternal request even at 25 weeks.

This is a controversial area, and again NICE’s experts wisely fudged the issue (again my italics): “Involve a senior obstetrician in discussions about whether and how to monitor the fetal heart rate for women who are between 23+0 and 25+6 weeks.” In my experience most UK experts advise against monitoring before 26 weeks, but even those who recommend it should also agree that Dr Laxman, who was a senior obstetrician, was working right at the limit of viability where even the best experts say there are no hard and fast rules. 

So why did the tribunal rule that not performing a Caesarean was negligent? I can’t read their minds, and perhaps they had further information, but I’ve sat on similar panels. It’s often politically difficult to say that an emotive tragedy was unavoidable. Easier to be wise after the event and blame someone. Dr Laxman, was in the frame because of what happened next. 

The baby didn’t deliver quickly. The body slipped through the incompletely dilated cervix and the head got trapped. Any obstetrician knows this is a horrible problem. For a larger baby the forceps can be applied to the trapped breech head, but not at 25 weeks and not with the cervix only 4 cm. Some experts advise incising the cervix but that’s a desperate, dangerous measure. Dr Laxman decided against. The baby died. 

Dr Laxman should have taken stock at this point. There’s no longer any clinical urgency, an oxytocin infusion could have been started and the baby would have soon delivered spontaneously. But emotions were high; the mother was awake, and perhaps the father was present too. They must have both been distressed. Assisting staff would also have been upset. Dr Laxman pressed on, urged the mother to push, and pulled again.

The head detached from the body. Extraordinary, but not unheard of, especially if a baby has been dead some time, although that situation did not apply here. Did anyone check whether the baby had some sort of connective tissue disorder? The newspapers don’t say. 

At this point other doctors got involved. They decided to perform a Caesarean to retrieve the head. Again we can criticise. Patience and an oxytocin drip, or slow removal through the vagina with the mother under a general anaesthetic and the father out of the room, would be my preferred option. But now I’m being wise after the event. Caesarean retrieval was reasonable.

In summary, a baby at the limit of viability died of natural causes, but a dreadful thing happened afterwards. The consultant involved, under pressure, pulled the head off. We don’t know whether she pulled too hard, or if some abnormality in the baby caused unusually weak ligaments. She had a previously unblemished record. No-one suggests it was anything other than an accident. It certainly wasn’t malicious. She did her best in a horrible situation. 

So what was gained by the public enquiry? Do the parents feel better? Will Dr Laxman, an ethnic minority doctor with a previously unblemished record – Dr Bawa Garba (click here) comes to mind – return to work? Baby decapitation headlines all over the world don’t make birth safer; they make sensitive doctors choose less demanding specialties, and the remainder defensively perform even more Caesareans. No lives are saved, and mothers are harmed. 

But newspaper readers had a pleasurable frisson of horror, and the mob had a chance to bay for blood. That’s something.

Jim Thornton

Advertisements

Canoeing the Loire – 2

May 31, 2018

Chateauneuf to Blois

Distances measured from Nevers (click here for Nevers to Chateauneuf). We camped at La Maltournee in Chateauneuf, and Val de Flux in Beaugency, but there are plenty of others.

No major obstacles. The rapid under George V bridge in Orleans is bouncy, and on a sunny day a tip would have an audience. But the rapids at Jargeau, Meung and Beaugency are all easy.

The weir at Saint Lauren-Nouant nuclear power station is not shootable. But it’s an easy safe portage.

No famous vineyards. The local AOCs are Orleans, Orleans Clery and Cheverny. AOC Cour Cheverny is special; white wine made in a few select villages from the ancient Romorantin grape found nowhere else.

132 km – Chateauneuf sur Loire bridge (1946) Town right. Campsite La Maltournee left bank 100 metres upstream of the bridge (click here). No local wine, but one of the few small breweries on the upper Loire, La Brasserie des Ecluses (click here) is a couple of miles up the road at Vitry aux Loges.

138 km – Jargeau bridge (1988)  Jargeau left. Saint Denis de l’Hotel right. Small rapid at site of an older bridge, downstream of the present one. Easily shot left or right.

Camping L’Isle Aux Moulins left after bridge (click here)

146 km – Checy right. Easy access.

    

Camping right (click here) (plan here plan_camping_municipal_checy) between the river and canal d’Orleans.

   

An adjacent campsite, also between the canal and the river, seems to have closed.

   

Alliance canoe rental opposite bank (click here)

148 km – Combleux right.  The canal d’Orleans connects by lock to the river here and runs along the right bank to Orleans.

   

      

The right bank is Orleans AOC (details here). A few locals are making an effort (click here).  Keep to the right side of the river through Orleans.

As Orleans approaches river traffic increases

152 km – Vierzon railway bridge (1947) followed by Pont Rene Thinat (1977)

          

Canal d’Orleans junction right. The Orleans terminus

    

This is the nearest the river gets to Orleans centre. i.e. not very near.

153 km -George V bridge (1763) Orleans centre right.  The rapid under the bridge deserves respect in an open canoe, but we got through without taking on water.

154 km – Marshal Joffre bridge (1958)

155.5 km – Europe bridge (2000)

Canoe club right. Good landing spot. Drains a bit smelly.

  

The campsite Gaston Marchand right about 200 metres upstream of motorway bridge (click here) seems to have closed (May 2018).

157 km – A71/E9 motorway bridge (1980)

158 km – Eglise Saint Mesmin right.  Les Forges right.

Camping Le Chateaux right, a few hundred yards after the church is an RV stop. No facilities.

159 KM – River Loiret joins left.  Easily missed. Carries little flow.

165 km – Saint Ay right. Camping la Fontaine de Rabelais (click here). Quiet when we paddled past.

       

The right bank AOC Orleans again. Left bank AOC Orleans-Clery (red cabernet franc). One of the best is Clos Saint Fiacre.  No vineyards along the bank. Most in or around the village of Mareau-aux-Prés.

orleans-clery

168 km – Meung-sur-Loire bridge (1948) Village right. Chateau Meung-sur-Loire (click here)

Shallows at site of old bridge/ford about 200 metres above the new suspension bridge. Shoot right.

Public swimming pool right bank just below the bridge.

174 km – Beaugency bridge (14th century)  Beaugency right. Camping Val de Flux left bank just upstream of the bridge. Click here for more. Rapid under the bridge. Shoot left or right.

         

Indoor and outdoor pool right bank just after the first island after the bridge.

179 km – Saint Lauren-Nouant des Eaux nuclear power station left.

Weir. Warning sign at 2km advises canoes to keep right. Ignore ambiguous sign right bank at 1km. Easy portage right just above the weir. Click here for more.

                 

181 km Camp friendship left.  Click here

186 km – Muides-sur-Loire bridge (1932). Camping Municipal Bellevue left bank below the bridge.  No website but it looks lovely. Easy landing left either before or after the bridge. The day we arrived a bric a brac sale was going on.

    

The left bank is now AOC Cheverney. Domaine Croc du Merle, farm and vineyard (click here) sells wine, cheese and other delicacies. But they don’t make or sell AOC Cour Cheverney. That is reserved for wine made in a few central villages from the traditional grape, Romorantin, found nowhere else.

       

188 km – Saint-dye-sur-Loire left

       

194 km – Cours-sur-Loire right.

      

196 Chateau de Menars right.

    

198 km – Disused railway bridge.

199 km – Camping left Val de Blois.  Activity centre, good access, campsite, canoe hire, pool, and this extraordinary structure. It is the Restaurant Le Lac de Loire. Some crazy architect decided the Loire needed this sort of brutalist concrete. But I rather like it.

         

We stopped here, so Blois itself will have to wait till next year.

Jim Thornton

Camping Val de Flux

May 29, 2018

Perfect base for Loire canoeing

I don’t usually post about individual campsites, but this spacious one just upstream of Beaugency bridge deserves the exception; an eleven out of ten.  Click here for the official site.

Good river access.

Super clean toilets and free hot showers.

The friendly barman speaks excellent English

The groundsman loves his job.

   

The goats praise God by being goats

Many other lovely touches

A few static tents

And loads of space, no unnecessary nonsense, and Beaugency over the bridge is lovely.

Jim Thornton

Saint Laurent des Eaux nuclear power station.

May 22, 2018

On the bank of the Loire

France’s third nuclear power plant opened in in 1969, with one Uranium Naturel Graphite Gaz (UNGG) gas-cooled reactor (similar to the UK’s Magnox reactors); a second was added in 1971. They were shut down in 1990 and 1992 respectively, but the housings remain, apart from the cooling towers, the largest buildings on the site, and the first things a canoeist sees.

In 1983 two 900MW pressurised water reactors (PWR), of similar design to the UK’s Sizewell B, came on stream and remain in service.  They are much smaller, and as trees have grown around the site, the concrete domes are inconspicuous from the river. In the left hand picture below they are visible in front of the right hand cooling tower.

The PWR reactors use water from the Loire to cool the steam after it has powered the turbines. Hence the cooling towers which condense most of the steam back to water for return to the river.

The cooling towers are visible from Beaugency. Two km above the power station a sign on the left bank warns of the forthcoming barrage. One km later an ambiguous sign on the right bank indicates that canoeists should keep to the right bank. Don’t misinterpret it as “land and start the portage here” or you’ll have a long carry.

The landing point is about 50 meters above the weir, marked by a proper portage sign, and easy to see and land at.

    

The portage itself is on a concrete path. Arrrgh! I’d forgotten my wheels.  With a sandy beach put in.

Here’s some views of the power station and the weir

   

  

A lovely spot for a picnic.

I love nuclear power.

Jim Thornton

River Thames

April 14, 2018

Hannington bridge to Lechlade

The river is canoeable from Cricklade but the level was high and we were nervous – if you’ve ever been caught in a strainer you won’t want to repeat the experience. No access at Kempsford. Hannington bridge has parking for a couple of cars, and access left bank above and below the bridge.

0 km – Hannington bridge

          

The next couple of bendy miles are tricky at high water.

Fallen trees need care. We portaged one at about 2 km.

3 km – The Thames path runs along the right bank, crossing inlets and the river Cole

     

4 km – Inglesham village and church right.

    

4.5 km – Inglesham Round House and old warehouse left. Followed immediately by the entrance to the derelict Thames and Severn canal, the mouth of the rive Coln, and a footbridge.

       

    

Moored boats above Lechlade

5.5 km Halfpenny Bridge Lechlade. Sadly now toll free. Land left immediately above for Riverside pub and canoe hire.

        

Or paddle through and land left for the New Inn

The New Inn is right in the centre of Lechlade

Click here for Lechlade to Northmoor, and here for the Oxford circuit.

Jim Thornton

Chasing the Modern by Tony S Hsu

April 2, 2018

The Twentieth Century Life of Poet Xu Zhimo

       

In 1915 the brilliant, privileged, 18-year-old Xu, was bought home to Xiashi by his family, from university in Beijing, and pushed into an arranged marriage with 15-year-old Youyi who he had never met. He got her pregnant but it was no love match and he soon took off without her on a world tour. Some years later, when he had made it to Cambridge, and fallen in love with another woman Lin Huyin, the family arranged for Youyi to leave her son in China and join him. Xu got her pregnant again but otherwise rejected her, and started leading a double life with Lin. Eventually he persuaded Youyi to grant him China’s first modern divorce; previously, high status men in Xu’s position would stay married, and take a concubine or two. The second child died young and Youyi eventually settled in Germany with her son.

Meanwhile the Lin Huyin affair ended with her also in an arranged marriage with someone else, and Xu returned to China, found fame as a poet and public intellectual, and began a scandalous affair with a glamorous married socialite, Lu Xioman.  Following another trip to England and the famous final goodbye (click here), he dragged Youyi back to China to persuade his sceptical family that she not only genuinely consented to the divorce, but was also agreeable to his second marriage to Xioman. Youyi complied, but the second marriage also rapidly disintegrated. Xioman partied all night, wasted their money, took a new lover and became addicted to opium.

Xu took the best way out, albeit involuntarily; die young in a plane crash! In the immediate aftermath his women reacted as expected. Xioman refused to believe the news, and Lin Huyin grieved, leaving the faithful Youyi to rescue his remains and sort out the funeral.

In the medium term his reputation faded under greater events; Japan’s invasion of Manchuria, the Long March, X’ian incident, and eventual Communist revolution.  His papers disappeared, Red Guards desecrated his tomb, and for 30 years he was written out of Chinese history. But in the 1980s under Deng Xioping’s reforms, his tomb was repaired, his diaries returned, and his reputation restored. His subsequent rise in popularity, driven largely by a couple of mass market TV soap operas based on his love life, has reached the point that no Chinese visitor to the UK can afford to miss the memorial stone on the banks of the Cam (click here).

Tony Hsu is the poet’s grandson by Youyi. This shows. Not just in the way his grandmother comes out of the story – although she was clearly a remarkable woman, one of the first in her family to avoid the dreadful foot binding – but in the occasional clunky platitude: “If evolution tells us anything it is that [our ancestors] intelligence and experience are lodged in our DNA”.

This is the only biography currently available for the non-specialist English reader. Short and padded out with pictures and biographical sketches of Xu’s famous friends, Liang Quichao, Bertrand Russell, Goldsworthy Lowes Dickinson, and Rabindranath Tagore, it also contains translations of sixteen poems. Enough for me.

Jim Thornton

Click here for “Leaving Cambridge Again”, and here for the Osmanthus poem.  Here’s another written six months before he died.

On The Bus

There are all ages and all trades on this bus:
Bearded men, unweaned babies, teenage boys, merchants and soldiers.
There are all the poses too: leaning, lying down,
Eyes open or closed, or staring out the window at darkness.

The wheels grind out refrains on the steel tracks;
No stars in the sky, not a lamp along the road,
Only the dim lights on the bus reveal the passengers-
Faces young and old, all fatigued.

Suddenly from the darkest corner comes
A singing, sweet and clear; like a mountain spring, a bird at dawn,
Or the sky lighting up over the vast desert,
Golden rays spreading to distant ravines.

She is a little child, her voice released in joy.
On this shadowed journey, at this dim hour,
Like a swollen mountain spring or a morning bird in ecstasy,
She sings until the bus is filled with wondrous melody.

One by one the passengers fall under its spell;
By and by their faces glow with delight.
Merchants, officers, the old and the young alike-
Even the sucking baby opens its eyes.

She sings and sings until the journey is brightened,
Until the fair moon and the stars come out from behind the clouds,
Until flowers on branches, like colored lanterns, vie in beauty,
And the slender grass rocks light-footed fireflies.

Xu Zhimo April 1931
Translated by Michelle Yeh

The $55M i3 trial of “Reading Recovery”

March 4, 2018

Results were biased by measuring outcomes when the intervention group hit their success target. 

Reading Recovery (RR), a method for helping poor readers, designed on the basis of research in the 1970s by New Zealand psychologist Marie Clay, is a worldwide movement. It is controversial, partly because it is a semi-secret commercial programme for which schools or parents have to pay, and partly because it emphasises “whole language” rather than phonics based methods.

In 2015 RR was evaluated in a large and expensive ($55M) randomised controlled trial in the United States. The report of the first year of the randomised element (click here ) is behind a paywall, but a copy is available here i3 reading recovery trial.  The full report of all four randomisation years, as well as the non-randomised scale-up phase is free (click here) or reading_recovery_final_report.

The trial has been criticised by some educationalists (click here) who appear to be more politically opposed to RR, than to have identified genuine flaws in the trial’s methods. They cite four “problems”.  1. Many low achievers were excluded. Answer: They were excluded from intervention and control groups equally, so did not bias the results. 2. The control group received a range of different experiences. Answer: Yes. They got “usual care”, common practice in a pragmatic trial. 3. The successful completion rate of students in the program was modest. Answer: Yes, but it still appeared to work. 4. No data supported the claim that Reading Recovery leads to sustained literacy learning gains. Answer: A valid criticism. Long term effects could not be measured in the randomised groups, because controls got RR at the end of the intervention period anyway.  Other critics have repeated the claims that the effect of RR was small but not sustained (click here) or focused on lack of definition of what the control group got (click here).

The trial also has defenders (e.g. click here). The What Works Clearing House, an independent outfit evaluating evidence-based educational interventions not only issued a special evaluation (click here) or wwc_may_102814, but gave the trial its strongest possible endorsement; “The research described in this report meets WWC group design standards without reservations.”

I’m a medical doctor with no vested interest – I only learned of the existence of RR a few weeks ago – but I do know a bit about randomised trials. So far as I’m aware, the more fundamental flaws identified in the rest of this post, have not previously been described.

Surprisingly for such a large trial, it was unregistered (the word registration or registry does not appear in the report), nor did it have a published protocol (the word protocol appears five times but never as part of “trial protocol”), so readers have to take on trust the statement that the primary and secondary outcomes, and sample size had been pre-specified.

It was conducted “from the 2011-2012 school year through 2014-2015”. 1,254 schools participated.  The research objective was “What is the immediate impact of Reading Recovery on the reading achievement of struggling 1st-grade readers, as compared with business-as-usual literacy instruction?”.

Population – “Struggling 1st-grade readers”. Staff in each school identified the eight pupils with the lowest Observation Survey of Early Literacy Achievement (OS) score.

Intervention – 30 minutes one to one Reading Recovery lessons per day for 12-20 weeks, as a supplement to regular classroom literacy instruction.  These are described as “[…] individualized, short-term, highly responsive instruction […] Lessons attend to phonemic awareness, phonics, vocabulary, fluency, and comprehension [… are] intended to help students develop a set of self-regulated strategies for problem-solving words, self-monitoring, and self-correcting that they can apply to the interpretation of text [and] enabling students to use meaning, structure, letter-sound relationships, and visual cues in their reading and writing.”

The RR lessons went on for at least 12 weeks, following which they stopped when the pupil reached the target OS achievement score of 16, or at 20 weeks, whichever came first. We will return to this point.

Control – regular classroom literacy instruction and access to any literacy supports that were normally provided to low achieving 1st-grade readers by their schools, other than Reading Recovery.  The researchers describe these in detail.

“We obtained information on 1,245 (57 percent) of all students assigned to the control group […]. 39 percent received no supplemental instructional supports; 37 percent participated in some individual or small-group intervention (other than Reading Recovery) provided by a Reading Recovery-trained teacher; 23 percent participated in a literacy intervention that was not delivered by a Reading Recovery teacher; and 8 percent received ELL or special education supports. Seven percent a combination of the services listed above. A majority of control group students (61 percent) did experience some form of supplemental literacy support in addition to regular classroom instruction. Therefore, in this study, we are comparing the effectiveness of Reading Recovery to that of classroom instruction plus a range of other support services that schools provide to struggling readers.”

This is clear. The trial is not, as some had alleged (click here), one of “”RR v nothing”, but of “RR v usual care”. However, the fact that the phonics/whole language components of usual care was not defined, mean the trial was never likely to contribute much to the phonics v whole language debate.

Outcome – The primary outcome was the Iowa Tests of Basic Skills (ITBS) at the end of the 12 -20 week intervention.  Secondary outcomes were the Reading Comprehension and Reading Words subscales of the ITBS, and the Observation Survey described above, which had been used to define the trial entry groups and to judge the length of the programme.

Randomisation – A total of 9,784 students were identified by picking the eight children with the lowest OS score per school. 1,254 times 8 = 10,032 but the missing pupils won’t bias the result because they were lost pre-randomisation.

“[…] teachers entered the names of the selected students into an online random assignment tool, noting their English language learner (ELL) status and their baseline OS Text Reading Level (TRL) subtest scores […]. The tool then matched them into pairs by first matching any students with ELL designations, then matching the student with the lowest TRL subtest score with the next-lowest student, and so on. Once the students were matched, a randomizing algorithm then randomly assigned one student in each pair to the treatment group and the other to the control group. The result was recorded in IDEC, and the tool was locked so that randomization in that school could not be redone.”

This is good. But then:

“4,892 students were randomized to treatment, and 4,892 to control. Both pretest and posttest data were available for 7,855 of these students (4,136 treatment; 3,719 control).”

This differential drop out (more drop outs among controls) would normally be a worry because drop outs might be poorly supported students. If you remove more poorly supported students from the control group than the intervention one, the control group would end up with higher scores, even if the intervention had no effect. Fortunately the researchers were aware of the problem and took steps to avoid it.

“Pairs in which either student was missing assessment data were dropped from the RCT, leaving a total of 6,888 students who were able to be matched into pairs with complete data (3,444 matched pairs in 1,122 schools).”

Excellent.  They did the right thing. If either member of a pair failed to complete the outcome measure they dropped both from analysis. As the authors put it, “Because the entire pair was dropped in the event that one student in a pair was missing outcome data, there is no differential attrition overall.” Here is the Consort flow diagram.

Table 2.3 (not shown here) shows the sex, race and baseline reading level of each group were, as would be expected, well balanced.

Now to the fundamental flaw; the timing of the outcome test.

“The precise timing of posttest [the short term outcomes] administration varied, as posttests were administered to both the treatment and the control student in a given matched pair immediately after the treatment student completed his or her 12- to 20-weeks of Reading Recovery lessons. This ensured that the two students experienced their assigned conditions for an identical time period. Typically, posttests were administered roughly halfway through the school year. As the study was designed as a delayed-treatment RCT, control students began receiving their Reading Recovery lessons after the posttests were administered to both treatment and control students in each matched pair.”

This sounds reasonable, but it biases results in favour of RR. Remember RR stopped when the pupil reached a score of 16 on the OS assessment, i.e at the point when the RR programme was judged to have succeeded, and the pupil could be returned to the normal classroom.  Since there will surely be some day to day variation in scores due to extraneous factors, the RR group are systematically being scored on, or near to, a good day.  Their control partner is measured on the same calendar day, which may or may not be a randomly good day for them*.

And here are the results. Table 2.3 and 2.4 are taken from the full (four year) trial report. For some reason tests of statistical significance were omitted from there – we’ve noticed before (click here) that some educationalists object to them – so on the right is table 2 of the first randomised year only trial report with its P values.

The top row (tables 2.3 and 2.4) shows the mean ITBS score at the end of the RR or control period. i.e. up to the point at which RR succeeded, or 20 weeks, whichever was sooner. The percentile ranks show where that score would lie on typical Grade 1 mid year scores. Rank 1 the worst reading centile, and 100 the best reading. In table 2 (right) the primary outcome (mean ITBS score) is now placed at the bottom.

        

The RR pupils had a mean ITBS score of 138.8 (high is good) and controls 135.4.  The mean difference of 3.4 is about half a standard deviation better (favouring RR). In the first year, with a quarter of the sample size, the scores were 139.2 and 135 respectively, also favouring RR by half a standard deviation, and very unlikely to have occurred by chance (P<0.001) (table 2).

But I’m afraid these results are biased. The only difference between the groups should have been the RR programme.  But, as we have seen, there was another difference.  The RR students were measured on a systematically “good day” but the controls on the same calendar day, which may, or may not have been a “good day” for them.  We need not concern ourselves over the possibility of data dredging, P-hacking, repeated looks at the data, or any of the other biases that can creep in to unregistered trials, because the raw data are already clearly biased.

It is extraordinary that the researchers, the What Works Clearing House, and all the trial’s critics, not only missed the lack of registration or a published protocol, but this obvious bias in the timing of the primary and secondary trial outcome measures.

It’s great that educationalists are doing randomised trials, but they need to be done properly.  Critics should study the methods carefully.

Jim Thornton

*FOOTNOTES – If you’re not convinced, consider a hypothetical pair of pupils, and that RR had no effect.  Imagine that both pupils were bumping along with OS scores between say 12 and 15.  Let’s further imagine that on week 15 the control pupil by chance had a good day and hit an OS score of 16, but the RR pupil, also by chance had a bad day and dropped to 11.  These two scores would not be noticed; the trial goes on.  Now imagine that the following week the RR pupil got things sorted out at home, did well and scored 16, while in his turn the control pupil had his share of home troubles and dropped to 11.  Because the RR pupil had hit the magic score of 16 the RR stopped and the triallists measured both pupils.  They are identical, each having had by chance one good day and one bad day, but if we measure them both on the RR pupil’s good day we get an RR data point of 16 and a control data point of 11.

The OS score is not the trial’s primary outcome, but it and the IELTS are both surely correlated with chance “good days”.

For the avoidance of doubt the subset of pupil pairs whose RR member battled on through the full 20 weeks are not biased, because they were each sampled at a point unrelated to either member’s achievement.  But only about half of pupil pairs went on for this long.

I guess that, given the trial dataset, a more expert statistician than I could measure the variation in OS scores, the proportion of pupil pairs who stopped RR for success before 20 weeks, and the correlation between OS and IELTS scores, and use these data to model the bias that was likely to have been introduced by this test-timing method.  It would be an interesting project.

%d bloggers like this: