Exams must go ahead in 2021

At Advantage Schools, we believe that national standardised assessments are important.

They are our best available method of assessing what pupils have learned whilst at school and therefore essential in better understanding the gap in educational achievement between different groups in society.

Exams are therefore important in our efforts to comprehend the impact of the pandemic on disadvantaged communities and to provide us with insight as to how we might respond as a result.

It doesn’t seem controversial to suggest that many pupils have missed learning as a result of school closures. Despite the incredible efforts of many teachers and leaders, this is likely to be exacerbated for the most disadvantaged pupils.

Pupils sitting examinations in 2020 and in 2021 have been disadvantaged in comparison with other cohorts.

In an attempt to mitigate some of this lost learning, governments in Scotland and in Wales have announced that examinations will not go ahead as planned. In England there are currently no plans to do so, thankfully. Depending on where a 16 or 18-year-old lives, they will now receive qualifications on the basis of their teachers’ assessments or some form of internally moderated qualifications. The argument is made that this is ‘fairer’.

This is wrong.

Assessment:

We should remind ourselves of the rationale for exams. We need to assess what pupils know and can do. Teachers do this formatively, to inform them of what they need to teach their pupils next, and summatively, so that they have some idea whether they are on track to becoming educated in the subject in question. Summative assessments work by testing a sample of pupil knowledge in order to draw a conclusion about their knowledge of the entire subject. Trying to understand if these are valid is very important. For more on validity, see all three parts, but particularly the second part of Daisy Christodoulou’s three-part blog.

As pupils end Key Stage 4 at 16, we need a proxy for what they know and can do. Good assessment needs to be valid – meaning it tells us what we want to know, which is why question choice is important – and reliable – which means it offers a consistent measurement and that the same pupil would get the same result over time, if they were assessed repeatedly. A set of scales that repeatedly gives the same wrong result would be reliable but not allow us to draw valid inferences.

It needs to do this so that we can be aware of how much pupils know compared to others, and for judgements made by employers, sixth forms and universities, for accountability, for motivation of pupils and so that as a jurisdiction we know how much pupils are learning for the money we spend on education.

Assessment needs to be accurate. That is as ‘fair’ as it can be.

Factors that compromise validity of assessment:

Human bias is one of the biggest factors in compromising the validity of conclusions we can draw from assessments. While teachers often know their pupils extremely well, counterintuitively this means that they may unwittingly jump to conclusions when assessing pupils. Indeed, teachers may have two or three grades they believe the pupil will achieve, and because we are an optimistic profession, we’ll generally predict that pupils will pull off the most optimistic of these. As a result, teacher assessment overall becomes weirdly skewed towards higher grades and while this may sound great, it hugely reduces validity.

Teacher assessment without examinations is a factor that compromises validity. In examinations, tasks are common, the marking is monitored and the awards standardised. Teacher assessment is prone to implicit bias and external pressure – for example from accountability measures. Around the world, in high-stakes arrangements which rely on teacher assessment, there are serious problems of systemic grade inflation.  This is hence highly likely when teachers mark examinations themselves. Furthermore, teacher assessment is demonstrably biased against marginal groups, including those from disadvantaged backgrounds.

If test conditions are not consistent, this will compromise validity.

Teaching to the test can mean that the pupil knows the sample that is being tested, but that the results are less valid because this is not an accurate indication of what the pupil knows in other parts of the subject. This is a concern when pupils are taught, for example, knowing which extract from a literature text will be examined – a pupil can learn an answer by rote, practice and get feedback from a teacher and if they are lucky also from a tutor, without even having read the entire text or having a sound understanding of literary conventions. A pupil may hence have a grade that suggests they would be more successful at A Level than they are in fact capable of being. Similar effects can be seen when a curriculum is geared towards the syllabus on a test.

All alternatives to exams significantly compromise validity.

Limitations to exams:

There are a number, including:

• A pupil may do better on one given day than another, or the content of examinations may be skewed towards what they are particularly good or bad at
• The high-stakes nature of examinations mean that it ceases to be a good measure of what it is attempting to measure (also known as Goodhart’s law)
• A pupil may just be a bit rubbish at exams. Exams cannot account for that. They are far from perfect, and even robust, well-designed examinations that are held in strict conditions cannot tell us with perfect validity what pupils know or don’t know
• There is also a debate to be had about criterion-referencing or norm-referencing the awarding of grades

Examinations are not perfect, even without a pandemic. In fact, examinations could be a lot better. But they are still better than all the alternatives – the solution to imperfect assessment is not no assessment.

The validity of the conclusions we can draw from examination results is the crucial benefit to examinations.

Validity of inferences drawn from assessments of pupils at 16 and 18 is the important thing to maintain.

Considering these limitations to exams, it is understandable that people should ask if there is a better alternative.

Alternatives to examinations:

Teacher assessment is less valid and reliable, as I’ve said. I’m also amazed that, following the mess of last summer, anyone would suggest going through that again, knowing the huge inbuilt grade inflation and with the additional pressure of those high-stakes decisions landing on pupils. On results awarded alone, last year’s cohort look like the best educated in recent years – even though their formal education ended prematurely with many disengaging long before exam season would have started – so it seems unlikely they are the best educated compared to previous years. If we can’t rely on grades to give us a true indication of how educated and knowledgeable someone is, what is the point is in awarding grades at all?

Clearly, those with resources to challenge, offer their children support or who are already advantaged in a multitude of ways are far more likely to be able to maintain their position were there to be no grades. This would cement the position of the better-off and ensure that, despite all of the heroic efforts of committed school staff, demography becomes destiny.

Awarding grades for what pupils don’t know, via some alteration according to how much learning pupils have missed, is likely to make validity so poor that the outcomes are useless.

One Headteacher suggested teacher assessments moderated by a sample of interviews with headteachers across the country. Obviously, this introduces variables that mean that validity will be impossible to tell. I think it might make us feel good while being ineffective.

Others have called for a combination of teacher assessment and examinations, but why do some pupils get valid results, and others not?

Some have suggested we should have no examinations and no qualifications. I have more time for this suggestion, but I think it will result in smaller-scale less valid assessments as admissions criteria, or could result in pupils in other years having a significant advantage over the 2021 generation.

My view, borrowed from people who know more than me, is that over the longer term we should note that Finland, the USA, Germany and Singapore all have the almost exact equivalent for A Level in their systems. The exams are almost identical – but in Finland and Germany more subjects are studied than examined. If there is a problem in England, it’s not that the exams are wrong and need to be changed; it’s that we have narrowed the way we think about the curriculum, building it almost exclusively from exams. 

There are no better alternatives to running GCSE and A-level examinations for 2021.

Should we change anything this year?

Probably. We cannot avoid that there are likely to be disrupted exams and a plan that considers significant learning loss.

The most obvious problem with examinations is being able to sit them while keeping pupils and colleagues as safe as possible. I do think “where there is a will there is a way” but in some centres this is going to need a lot of goodwill. It may mean a shorter period for examinations, with schools partly closed for a very short period, or closer collaboration between centres.

As well as this, pupils who are self-isolating obviously need to be awarded grades. We should consider second sittings, spacing out some examinations so that we can extrapolate a (relatively) valid assessment from a single paper, and we should look at all other options.

I know that Ofqual are looking into possible alternatives should the pandemic be widespread in 2021. I know there are a number of smart people doing so determined to avoid a repeat of last year, and I believe they will be sensible. The Education Policy Institute also offers some smart suggestions. I understand that the precise mechanisms for this will be agreed by end of November, and consulted upon. Exams boards, Ofqual and Her Majesty’s Government are working hard on fair, practical adjustments to exams. 

Losing examinations is the very last thing that should be considered.

Schooling is unfair in 2020:

In some schools, up to 60% of the pupils in Year 11 or Year 13 have missed school.

We shouldn’t downplay the efforts of schools and teachers to provide an online curriculum that ensures that pupils make progress, alongside ensuring that pupils and families are as safe as possible, while also running a school with pupils physically present. However, very few people would argue that online provision can match face-to-face provision. Therefore, in those schools, pupils are likely to do worse in national standardised examinations.

The instinct might be to lower standards for these pupils, or ask teachers to award grades that are not dependent on what pupils know and can do. On what basis should they award these grades? On what they think pupils know or can do? Well, we’ve already established the most valid, unbiased way of establishing that is through an exam. So, if we are arguing against the more accurate, exam-based measure, it seems we are asking for teachers to assess based on their perception of a pupil’s potential – what they guess they would have achieved had there not been a global pandemic disrupting their education. 

We get into real difficulties when we start to try to work out what pupils could have achieved without this disadvantage.

We would have no idea about what pupils can actually do, nothing valid to use in the future, and have undermined all the awarding of grades.

Things have been unfair this year, and most unfair for the most disadvantaged.

Schooling is unfair in every other year

Every year all pupils sit the same examinations in broadly the same conditions at the same time.

Some of those pupils are taught in great schools. Some are taught by great teachers. Some of them had been taught in small classes. Some have teachers as parents. Some have private tutors. Some of them have other additional tuition. Some of them don’t have English as a first language. Some of them are in schools where they can’t concentrate because of poor behaviour. Some of them are in poor schools. Some of them are in schools that cannot recruit in particular subjects, or at all. Some of them have teachers that are ill. Some of them are in homes where the main concern is the next meal. Some of them are in rural schools with huge transportation issues.

Education and schooling is unfair. This is why Advantage Schools exists and our aim is to mitigate this.

Exams don’t mitigate disadvantage, and nor should they

Examinations do not solve the endemic issues of poverty and disadvantage. They don’t close the achievement gap. They don’t award higher grades for those who have a more significant disadvantage. Part of the reason is that if they did, they would not be accurate. And if they aren’t accurate, they aren’t valid.

If we use exams to mitigate disadvantage, we might be awarding disadvantaged pupils higher grades, but we will not have taught them a single thing more, and the widened education gap will continue to exist. Our efforts will have covered up this gap. We are being, perhaps partly unintentionally, dishonest about its existence.

So we should sit examinations, be honest about the results and the deficiencies in learning, and place the effort in ensuring that admissions departments and employers are aware of the compromised situation in 2021 and not undermining the credibility of the best form of assessment that we have.

There are many different ideas of what it means to be fair in assessment, and in current times some of those are colliding. What we must not lose sight of is that exams need to be an accurate measurement of what a person knows, understands and can do.

Assessments need to be accurate.

How do we mitigate disadvantage?

I believe the best way is to have the best thought-through curriculum that is as aspirational as possible, and that we need to deliver it as best we can to pupils. I think we need to embrace the challenge to do this significantly better than is the norm, or average, at the moment. But there may be better bets that we can make with the limited time and resources we have available which include:

• Providing digital devices to families without access
• Funding schools with higher levels of disadvantage significantly more, and allowing schools to spend this on their best strategies
• Even more focus on Oak National Academy
• Funding schools to have the best subject-specialist curricula
• One-to-one tuition where required
• Collaboration (including between private and state institutions)
• Extending free school meals
• Reconsidering the high-stakes nature of the examinations for institutions – so that we can maintain the impetus towards ensuring pupils are educated without having leaders and teachers fear for their jobs – this could include strong guidance to the media not to manufacture league tables or a suspension of the individual school examination figures

I’m sure there are lots of others. This needs to be our focus while keeping exams.

We should not pretend to have mitigated disadvantage by manipulating the best measure of assessment.

2020

Some have argued that examination grades are merely an indication about the potential of pupils. They are used for this purpose, but that is because the amount that pupils know and can do now is the best indicator of what they might know and can do in the future. But they are an assessment of current knowledge.

When Centre-Assessed Grades (CAGs) were awarded in 2020, pupils at our secondary school who had done well looked relatively downcast, even on results day. There was a sense that it wasn’t real, that they hadn’t been able to show what they could do, and that they hadn’t achieved validation.

A few months earlier, I’d had to stand in front of them telling them that they wouldn’t be sitting examinations. None of them were pleased about it.

In many schools, I heard about engagement in year 11 “falling off a cliff” and about the significant gains in learning that can be made at the end of a course – when in some subjects over the last few months the components of knowledge and skills that have been built by really careful curriculum cohesion that come together as a composite of performance in the subject – that final composite didn’t materialise. The lack of a decisive end-point made this extremely difficult, and set pupils even further back than the loss of face-to-face time.

Furthermore, in 2020, CAGs were significantly inflated in some schools. As a result, one of the real limitations of using CAGs again is that nobody will trust them. We might as well not bother at all. And what this will mean is lots of other more oblique criteria will be used by destinations to make decisions between candidates. This will disadvantage the disadvantaged most.

We cannot repeat the chaos of 2020.

This is high stakes

If the current year 11 or year 13 don’t sit examinations because they have missed a certain number of months of their education, then the precedent is likely to mean that next year’s GCSE and A-Levels may not take place either, due to Year 11s and 13s having missed at least the same amount of education. And if they don’t sit examination, why should anyone sit them in 2023?

Of course, many of the loudest arguments against exams are not being made as a result of the current challenges of the pandemic. They are well-rehearsed arguments by campaigns to try and remove examinations from our education system, rooted in an ideology that believes that there must be a better way, but which hasn’t yet found an alternative. These voices are capitalising on well-intended concerns about “fairness” to try to achieve their goal. The current febrile environment amplifies this voice. But sober consideration of events and the needs of society and individuals shows that this voice fails to solve the problem it identifies. Indeed, it exacerbates the problem.

The irony is that the removal of GCSE and A-Level examinations could leave us with a system that entrenches disadvantage further, and only enhances the devastating impact of the pandemic on children and young people in future years.

An informed debate on how we could further the reliability and validity of examinations across our school system is welcome. But any claim that exams should be removed should be accompanied by robust and plausible proposals for an alternative which doesn’t have negative unintended consequences for those we want to help the most.

The fairest thing we can do for this generation is to judge them on the same standards of others and then do what we do best to help fill the gaps. Anything else isn’t taking responsibility.

That is why the very last thing that anyone should argue for is the removal of GCSE or A-Level examinations in England this year.