The power of randomisation

Mistake in my P4C-2 post yesterday (click here). The result is still negative, and the report still turgid, but when describing the distribution of the pooled reading and maths scores, I suggested that the authors had not shown them by group, and made the snide comment “Perish the thought that an educationalist would ever show you anything remotely near to the raw data!”

Forgive me. Further down appendix I of the main report (click here) are “intervention” (p 100) and, controls (p 102). But different pages? This is a randomised trial where half the pupils got philosophy teaching and half did not. Readers want to compare the two groups side by side. Come on EEF!

To help I’ve reordered the graphs here p4c2 result histograms p number. Intervention group left, control right. Page one, free school meal (FSM) pupils and page two all pupils. Reading scores before maths scores.

It’s rather revealing. For each outcome and for each subgroup not only are the means and ranges almost identical but also the shapes of the distributions. Not exactly identical – these are different populations – but remarkably similar. Go on, check. Click the link above and then scroll down comparing left to right. The shape of the control distribution of reading scores among FSM pupils is different from the shape of the control maths scores, but both are almost identical to the respective reading and maths scores for the intervention group.  Same for the whole sample.

Not only did the intervention, Philosophy for Children, have no effect on maths or reading but, since it had no effect, we can see how beautifully, when you have a large sample size, randomisation really does generate comparable groups.

Well I think it beautiful.

Jim Thornton