#53ideas 38 – Clearly stated goals are not always helpful to students

John Biggs powerful idea (No 37) is in part a new take on an idea that has been around for a very long time: that the most important part of planning education is deciding where you want to get to. When I first started out as an educational developer this took the form of specifying ‘behavioural objectives’, written in the form “At the end of this unit students will be able to…” and then specifying a verb such as ‘”define”, “recognise” or “calculate”. Behavioural objectives were about what you could see and verify students actually doing with whatever they had learnt – about their visible behaviour, not about their invisible thinking. When I joined the Open University in the 1970’s every course unit on every course had a list of 20-30 such behavioural objectives, even in Arts courses, and there were 12 units on a course so that was up to 360 objectives! The Open University eventually abandoned this practice and I’d like to discuss why, and why such detailed specification might sometimes be less useful than often believed, or even unnecessary.

First, let’s start with behavioural objectives, and then move on to the kind of specification that we see most often nowadays: learning outcomes. Behaviourism came out of a reaction within Psychology, more than half a century ago, to the unreliability of ‘introspection’ and the difficulty of knowing with much certainty what anyone is thinking or knowing or experiencing unless you can actually see them do something based on their thoughts or their knowledge. It resulted in extraordinarily detailed specification of every school classroom session, each day, each week, in each course. The lists of objectives became vast and unwieldy and led to colossal testing regimes attempting to measure the achievement of every objective. They were also often much too low level – low level ones were easy to write and test while high level ones, about more sophisticated educational goals, were very difficult to capture in the language of behaviour and were difficult to distinguish in terms of their level. They were also very expensive to assess. As a result high level behavioural objectives became an endangered species. I remember attempts by Oxford Polytechnic Lecturers on BTEC courses to write objectives for their courses, which they were obliged to do, and I did an analysis for the course review committee of the level they were at. Almost always far too many were low level objectives (about remembering facts or using specified procedures) and understanding, creativity, problem solving, analysis and other less easily specifiable processes got dropped. The level of objectives in BTEC courses were meant to increase as the level of the course increased, but they didn’t – all objectives were actually much the same regardless of the level of the course, and dumbed down what students were meant to learn and what they were tested on. It turns out that my observations were far from new – this was an almost universal phenomenon.

It also turned out from research into the use of behavioural objectives that students often did not understand the objectives or ignored them. I have interviewed students who studied highly specified curricula and who had, after three years’ confusion, given up trying to make sense of objectives altogether and trying to cope with variations in the meaning of words such as ‘analyse’ between courses and between teachers, and decided that they would simply learn what they thought best.

Then it emerged from research that teachers didn’t understand behavioural objectives either, and different teachers understood the same objectives in different ways. More detailed specification seemed to make things worse rather than better. Different teachers would write different objectives for the same curriculum and have no way of agreeing which ones to use. Even quite small courses seemed to need hundreds of objectives, and could generate hundreds of possible alternative objectives with little to choose between them. Where objectives came from, and which were the ‘right’ objectives, became quite a problem without recourse to values and intentions that are extremely difficult to specify behaviourally.

It could be argued (and usually is by those who are fond of objectives and outcomes) that the problem here was with the competence of teachers to set objectives in the prescribed manner, not with the general idea. But even with training and practice and scrutiny by experts, the form objectives took was seldom impressive. Perfection in this was achievable.

Then there was the side effect on teaching that is narrowly constrained by long lists of objectives – it was often dead on its feet. Experienced teachers had good ideas about what engaged their students and what led to most learning, in a general kind of way, but felt they could not use this experience if they were obliged to keep narrowly to lists of pre-specified objectives. Interesting and challenging learning activities are often associated with exactly the kind of higher level objectives that were more difficult to write or test. Classrooms became dull lifeless affairs and testing became routine and boring. I do not believe it is a coincidence that students study fewer hours and are less engaged in America, where they are still in thrall to rational curriculum specification. Much good teaching flows, I believe, from teachers’ repertoire, developed through painful experience, of designing learning activities that they have found to engage students, and a dumped repertoire of activities that they have found do not work. Good teachers know which learning activities are rich with potential for learning and which are very limited. Much effective course design, I believe, starts from this knowledge about fruitful learning activities, rather than from a rational analysis of what kind of activity is most tightly linked to what objective. Sometimes it is possible for a teacher to think “I know what kinds of activities work, so I am going to make up some objectives that appear likely to be achieved by my activities, and pretend I started with the objectives.”

Despite the apparently self-evident truth that if you don’t know where you are going you are unlikely to get there, gradually almost everyone in higher education, outside America, gave up on behavioural objectives.

Then there is the issue of learning things other than what is specified in the course document. Most students, I believe, leave higher education as changed people. They read different kinds of things, they are interested in different issues, they are more aware of all kinds of things, they have different discussions, they argue differently and more logically, and so on. Try finding these things in course objectives! Within individual courses students learn course content that is not written down in course documentation. I remember as a student becoming interested in some topics that were peripheral or missing from the formal curriculum and learning quite a lot about them because I was interested. If it were possible to map what I learnt overall against what I was supposed to have been taught, then there would have been some overlap, but there would be lots of things I was supposed to learn that I did not, and lots more that I learnt because I was supposed to – and promptly forgot -and then a whole realm of other learning that my course did not intend and that my teachers did not know about at all. And it was probably different for every student. Unintended or incidental learning sometimes seems to make up quite a large proportion of all learning, and I believe that attempts to specify learning too tightly, and to test too narrowly to these specifications, both reduces this unintended learning and undervalues it. I remember a final year project in Archaeology at Oxford. It was huge and took students much of the 2nd year planning it and much of the third year undertaking it. Students could basically do whatever they wanted – in the lab, in the library, in the field. It would have taken some imagination to draw up a set of objectives that spanned the range of learning activities and topics that different projects involved: carbon dating, numismatics, linguistics, economics … And then the tutors did not even put a mark on the project report! Feedback yes, marks no. When I asked for a rationale for this it was all about engaging students – that academic learning flows from interest in a topic, and they wanted to give students a chance to get fully engaged with whatever interested them. Learning of roughly the right kind would flow from this, they argued, and marking would spoil everything as it would cut across students’ interests and risk-taking. It was clear from external examiners that students’ projects were of an extraordinarily high standard and many of the students went on to graduate work and academic careers, but there was no specification of objectives at all, let alone any assessment of their achievement. Because Oxford, and its students, are so atypical, I do not want to argue for any generalisable truth based on this example, but a great deal of very high level learning is nevertheless clearly possible whilst entirely ignoring the whole business of specifying objectives.

Now lets move on to the current use of ‘learning outcomes’ which is more the kind of thing John Biggs has in mind in his case for alignment between goals and learning activities. In general I believe he is right, that to design a course so that it involves the kind of educational activities that are likely to lead to intended outcomes, you need to have a clear idea in your head of what these outcomes consist of. I’m sure that the Archaeology dons in the example above could have articulated some sensible goals, if pressed to do so. The issue I want to discuss here is whether this idea needs to leave the teachers’ head in the form of learning outcomes written down in course documents, or even whether it is possible to write outcomes down in a way that is useful to students.

One research study I undertook involved looking closely at how students responded to their assessment regimes, at three very different kinds of university: Oxford, a new University that used to be a Polytechnic, and an adolescent research-intensive university. Part of the study focussed on the extent to which students understood what they were supposed to be learning and what counted as success. We used questionnaires developed for the purpose, in-depth interviews and focus groups in three disciplines in each university. We found a strong inverse relationship between the level of specification of learning outcomes and assessment criteria on the one hand, and the extent to which students were clear about what they were supposed to be doing on the other. Students at Oxford were usually very clear despite the fact that most courses did not have learning outcomes at all, let alone assessment criteria. Where courses did specify ‘outcomes’ they were often little more than the statement that “Students will learn about…” and a list of the curriculum content in the old fashioned way. What is more, outside Oxford, where learning outcomes and criteria were specified in careful detail, students tended to narrow their focus of attention to what was specified (if they could make sense of it) while Oxford students had a much broader sense of what was interesting and worthwhile to focus on and were directing themselves, putting in far more study hours than the students at the other universities. The question then emerges is how Oxford students could be so clear about what they were doing when almost nothing was specified. The answer seemed to lay in the extent of close contact with teachers, the frequency of very small group tutorials (where they heard the tutor commenting on other students’ essays that had either hit or missed the mark) and the enormous quantity of formative assessment and feedback. They learnt what they were supposed to be doing through interaction and experience, gradually becoming members of the disciplinary community and becoming familiar with its discourse and values. The students we studied were third year students and it is a common observation at Oxford that it takes many students much of their first year to work out what the hell is going on and what they should be doing when they ‘study the subject’. But if it takes very bright, able and hard working students a year to understand this when they have masses of close contact and feedback, how could reading a list of learning outcomes manage this? The answer, I think, is that it could not, and does not. Students do not understand bald statements of learning outcomes – or rather they understand them partially and in different way from each other and a different way to their teachers. I think the clarity of learning outcomes is a mirage. To understand them you need to be able to use the discourse of the discipline and you need prerequisite knowledge – and lots of experience of using that discourse and knowledge.

I suspect that if teachers were to follow John Biggs’ guidance, and design well aligned learning activities and learning outcomes, which seems very sensible, then students would be more likely to understand the intended outcomes by engaging in the activities, rather than by reading the list of outcomes.

This blog post is also available as a PDF

---

53 Powerful Ideas All Teachers Should Know About

SEDA is publishing online both on its website and on its blog one of Graham Gibbs’ 53 Powerful Ideas All Teachers Should Know About, with the intention of prompting debate about the underlying basis of our work. Graham has also invited a number of well respected international thinkers and writers about university teaching, and how to improve it, to each contribute one idea to the ’53 Powerful ideas’ collection. On the blog we hope that you will comment on and discuss the ideas set out. Once all 53 ideas have been published, the intention is to hand over to our community and publish one idea from members each week, to continue the debate.

We invite you to join the discussion using the comment box below in response to Graham’s question.

How do your students come to understand what your course intended that they learn?

What proportion of what your students learn is adequately defined by formally stated learning outcomes?