Published on: 22 Feb 2016

Peter Kent, a past president of ASCL, has now famously described P8 as “the least bad measure so far”.

Five questions about Progress 8

By Duncan Baldwin, Deputy Policy Director, Association of School and College Leaders

1.       Is it fair?

Yes. Assuming you are talking about it in relation to prior attainment at pupil level across the whole population it is perfectly fair, indeed you would be right to say it is statistically unbiased if you were so minded. However, for the usual sub-groups of pupils that you usually worry about it is biased, in that the average P8 score for those groups is negative. Very helpfully, RAISEonline interactive lets you look at P8 just for those groups compared with their peers – it’s definitely worth a look if you haven’t yet engaged with it.

At whole school level, any value-added measure advantages schools with higher levels of attainment and Progress 8 is no exception. The only real way to treat schools fairly is to use contextual value added, but I can’t see a return to that any time soon.

2.       Is it a good thing?

Peter Kent, a past president of ASCL, has now famously described P8 as “the least bad measure so far”. Praise indeed.

ASCL has welcomed P8 because:

  • It recognises progress at all grade boundaries, not just the C/D borderline
  • Whilst it features English and mathematics quite heavily (they are double weighted), other subjects can and do contribute to it significantly
  • Schools close to my heart, which make great progress with disadvantaged intakes, can do really well against this measure. It doesn’t reward intake in the same way that 5A*CEM or Ebacc have. In particular, a school’s floor target is a function of the pupils it has, rather than an arbitrary line in the sand.

3.       Are there any bad points?

Yes. No single measure is perfect (which is why the government is right to want a range of headline measures, even though I believe it hasn’t chosen the right set). The bad points of P8 are:

  • It’s hard to calculate 
  • It’s arcane, so very few stakeholders will understand it. I’m quite sure some parents will think -0.3 is better than +0.1 for example, and I’m pretty sure explaining statistical significance to them will be a non starter.
  • It’s vulnerable to outliers. Your worst nightmare is a boy entitled to free school meals predicted to get 8 A* at GCSE who breaks his leg the day before the first exam. One pupil like this can really drag the P8 score down (again interactive RAISE will help you see this effect).
  • You won’t really know how well you have done until performance tables come out. It was easy to work out your 5A*CEM on the day of results, but P8 depends on everyone else’s results. (And goodness knows what schools will put on the banners they tie on their school railings – one of our members suggested reusing the one they had that said “Best results ever”!)

4.       What should we be doing about it?

There are only two things you can do to improve your Progress 8 score:

  • enter pupils for subjects that match the subject mix in the measure
  • teach better

One thing you definitely should not be doing is setting targets for Progress 8. No-one knows for sure how it will behave year on year so you are better advised spending your time on things that you know make a difference and would change anyway if you have a low score, like better teaching. If you must, use Attainment 8 for your target setting, which is much more predictable.

5.       Will it survive?

This is a good question. The DfE have recently committed to using the measure up to 2018. A serious weakness of P8 is that entries to Modern Languages are falling, and they can be “avoided” in the Ebacc element of the measure (two sciences and history would do the trick, for example). The government is right to be concerned about this and is one reason why it has emphasised increasing Ebacc entry over the course of this parliament.

At ASCL we think this is a much more complex problem to solve than simply manipulating performance measures and is why we have recommended a fundamental review into MFL, including curriculum, pedagogy, recruiting teachers and its treatment in performance measures. One way to help retain the benefits of P8 and address this would be to rebalance the subject mix to include four Ebacc subjects but only two other subjects in the fourth element.

Progress 8 encourages equitable treatment and attention for all pupils. Threshold measures like 5A*CEM and Ebacc take us backwards. Let’s hope it is here to stay.

 

Follow Duncan on Twitter @Duncan_Baldwin

Assessing students with EAL

Sue Thompson talks about the different approaches to assessing students with EAL.

Using computerised assessment with SEND children

Jo Horne explores the advantages and disadvantages of using computerised assessments with special educational needs (SEND) children.

Girls with social, emotional and behavioural difficulties

John Galloway discusses how we can identify and support girls with social, emotional and behavioural difficulties.

Working with families

Educational Psychologist Poppy Ionides discusses how we work with families to improve outcomes for at risk children and fragile learners.