Until recently in the UK national league tables that purported to show which were the ‘best’ universities for students to apply to included the proportion of students who gained a ‘good’ degree: a first class or an upper second class degree. This proportion varied widely between institutions (from about three quarters of all students to about one quarter) and was assumed to tell you something important about quality: if more students performed well then their education must have been better, it was argued.
There are reasons to be sceptical about degree classifications as measures of student performance. There are large differences between institutions that have no obvious explanation: there appear to be consistent historical norms. There has been massive grade inflation and the extent of grade inflation has varied according to the type of institution and has been particularly rife amongst the research élite, who report the highest proportion of good degrees. This grade inflation has taken place while a number of variables that negatively predict student performance, such as class sizes and the proportion of teaching undertaken by part time teachers, have got much worse: so grades should have gone down, not up. If you add up marks from courses within a degree programme using the regulations and formulae from a range of institutions then you get a range of degree classifications. Degree programmes with a preponderance of coursework assessment do not assess the whole curriculum, but only the topics students addressed in their chosen assignments, so degree classifications are then not a reliable indication of student learning across the whole curriculum. In contrast unseen exams that sample the curriculum unpredictably do provide a reasonable indication of student learning across the whole curriculum. Coursework-assessed courses inflate apparent performance, and there has been a substantial shift to coursework-assessed courses. And so on. A Parliamentary Select Committee concluded in 2010 that they had no confidence in standards in English Higher Education – not “a few concerns”, not “limited confidence”, but “no confidence”. As far as they could see, degree classifications told you virtually nothing you could rely on.
But the main problem with student performance data is not that it is problematically and systematically unreliable, but that it is predicted primarily by the quality of students entering the institution. In the USA High School grades can predict up to 90% of the variation in college grades. There is sometimes little going on other than that the most selective institutions produce the best student performance, independently of how they teach. The relationship between A-level scores and degree classifications is much less close in the UK, which is an illuminating phenomenon, but the relationship is still substantial. You can predict student performance equally well by looking at institutions’ research prowess or resources, but this is simply because the rich high status institutions attract the best students, who then perform best. These rich high status institutions are known not to use pedagogic practices that have been found to improve student learning to any greater extent than do poor and lower status institutions, underlining the problem of using student performance as an indicator of educational quality
If you want to understand which institutions (or degree programmes) work best pedagogically you have to find a way to measure learning gains: the difference between what students know and can do when they enter and when they leave. Once you do this it is possible to spot which pedagogic practices achieve better learning gains, and you can spot who uses these practices most often and hence predict how much students will learn in different contexts. And you do not need to know anything about how research-oriented or selective the institution or department is. Learning gains are predicted by educational processes, while performance is largely predicted by ‘presage’ variables: aspects of the institution that describe the environment before any teaching or learning takes place.
The things that best predict learning gains are close contact with teachers, collaborative learning, good fast feedback, time on task, clear and high expectation, and taking a ‘deep approach’ to learning. As pointed out above, these things are not found any more frequently in highly selective institutions that recruit the best students and produce the best performance, and so learning gains are no greater in the research élite.
League tables should not include either A-level points scores or the proportion of students gaining good degrees, if the intention is to identify where the best education can be found. Rather they should include indicators of the extent to which effective pedagogic practices are used and how engaged students are as a consequence of experiencing these practices.
53 Powerful Ideas All Teachers Should Know About
SEDA is publishing online both on its website and on its blog one of Graham Gibbs’ 53 Powerful Ideas All Teachers Should Know About every week for a year, with the intention of prompting debate about the underlying basis of our work. On the blog we hope that you will comment on and discuss the ideas set out. After a year the intention is to hand over to our community and publish one idea from someone other than Graham Gibbs, each week, to continue the debate.
We invite you to join the discussion using the comment box below in response to Graham’s question.
On your course, what best predicts how much students will learn?