#53ideas 27 – Making feedback work involves more than giving feedback – Part 1 the assessment context

As student numbers increase there are some economies of scale to be achieved in teaching methods (even if this only involves larger lecture classes) but assessment costs increase pretty much in proportion to the number of students. A consequence is that at some point assessment costs (in terms of teacher time) overtake teaching costs. I have come across contexts where the assessment costs very much more than the teaching. But much of this lavish investment in assessment, largely spent on teachers writing feedback, is wasted because students don’t read the feedback, or make very little use of it. National Student Survey scores on feedback are usually amongst the lowest for any aspect of students’ higher education experience, despite all the investment.

Much of the effort to improve feedback on students’ work, and to improve National Student Survey scores, focuses on the feedback itself – what teachers write down on assignments. However this item is not about what teachers write, but about the context in which the feedback is provided. You can write perfect feedback and it still be an almost complete waste of your time. Getting the context right usually makes more difference than putting extra effort into the feedback itself. There are so many issues involved here that this item is divided in two. Part 1 here addresses issues about the design of assessment systems and the markers. Part 2 addresses issues about the students, their motivations, how they perceive feedback and what they do with it.

Timing

Students interviewed about the value of the feedback they are getting often say things like “I didn’t even read it – it’s just water under the bridge, isn’t it?” One of the ‘Seven Principles of Good Practice in Undergraduate Education’ is “Good practice provides prompt feedback” and the key work here is ‘prompt’. If students are already tackling the next assignment, or if the course is effectively over, then the feedback is likely to be perceived as useless. Responding to student requests for full written feedback on exam answers seems a particularly poor use of resources. In practice, quick and dirty feedback (such as model answers, generic feedback on a sample of the cohort of students’ work, and peer feedback or discussion) can work much better than slow and perfect feedback – it has to be fast enough that students are still interested. What matters is the quantity and quality of students’ reflection on their own work and their learning, and such reflection can be prompted in all kinds of imaginative ways at very low cost. University policies to ensure that all feedback is provided within three weeks seem feeble – this is often very much too slow.

Feedback also has to be provided in time so that that student can use the feedback. And that often requires not just promptness, but redesign of the sequence of assignments (see below).

Marks

Experimental studies have provided students with one of three feedback conditions: marks only, marks and feedback, or feedback only. It is the feedback-only condition that produces more learning and more improvement in student marks. Students demand marks, but what do they know? Taking marks off assignments is the cheapest way to improve the impact of feedback. If necessary you can provide marks two weeks later once the feedback has been digested. Students’ extrinsic motivation and mark orientation can make even the most lovingly and professionally crafted feedback metamorphose into an explanation of why they got the mark they did. This does not necessarily help them to learn the subject matter.

Most universities provide marks, or marks plus feedback, about ten times as often as feedback only. At the University of Oxford it is the other way round, with feedback (usually on tutorial essays) about ten times as frequent as marks (usually on exams, and usually only right at the end, after all the learning from feedback has taken place). This works much better to support learning. The most powerful way to improve feedback would be a significant shift from summative to formative feedback, and to feedback-only assignments. If you have scope for four assignments on a course you could, for example, provide feedback on the first three and marks only on the fourth. It could also make Examination Board meetings a lot shorter. At Oxford such Boards may only have a dozen marks to consider, to determine a student’s degree classification while elsewhere it can be a hundred, and the most I have ever come across was over 500.

Assignment sequence design

Students are most likely to pay attention to feedback, and to use it, when it can help them to do the next thing. Perhaps the best example of this is feedback to doctoral students on draft chapters of their thesis. It would be a very silly, and unusual, student who did not revise and improve their drafts in light of the feedback. Two stage assignments in which only feedback is provided on a draft and only marks are provided on the final submission invariably improve student performance.

Feedback also works well when there is a sequence of linked assignments: for example assignment 1 feeds into 2 which feeds into 3 and then assignment 4 pulls them all together. At each stage feedback would help students to tackle the next thing they do. Even if the content of assignments is not linked, if there a series of the same kind of assignment – for example all lab reports or all case studies – then feedback about the way the assignment was tackled can, if configured appropriately, help students to tackle the next one more successfully. The assessment design that works least well is ‘one cycle’ assignments – for example a single essay with nothing linked to it before or afterwards. Students might well say “I don’t care if the marker has gone to the trouble of writing lots of feedback. I am not going to do another one of those any time soon!”

In modular course structures, and even in parts of single subject degree programmes that have ostensibly been design as an integrated whole, it is common for course content not to be encountered again in later course units. In interviews students have said things to me such as “Why would I read the feedback? I am never going to use this stuff ever again” If anything gets carried forwards in such contexts it is more generic learning outcomes such as how to tackle projects, how to write, or how to present quantitative data clearly. Those giving feedback need to be aware of what comes next for students and what aspects of what they are marking carry the potential for ‘feed forwards’. Markers who have no idea what comes next cannot do this (see below ‘Employment practices’)

Assessment congruence and authentic assessment

Students sometimes make comments like “Why are we being asked to do this? I don’t see the point” “What are we supposed to be learning here?” or “What use is this feedback when I cannot make any sense of the assignment?”

Students may be told that the reason they are writing up so many lab reports is so that they learn about experimental design – but every single lab has been designed by the teacher (or more likely by a different teacher some years ago) and neither the labs nor the exam questions actually ask them to design experiments. Students may be told in a course guide that the rationale of a philosophy of education course is that philosophical concepts can help teachers to recognise classroom dilemmas and think their way through them better. Then they are set essays in which they are asked to recite what they know about an 18^th century French philosopher. It is hard for students to make sense of feedback, or to value it, when the assignment they have been set is not congruent with the specified learning outcomes of the course and the feedback is not even intended to help them to achieve these outcomes.

Assignments that make sense to students are sometimes called ‘authentic assessment’ in that they resemble worthwhile real world problems, challenges or tasks that their learning is designed to help them to tackle. On many courses no thought has been given to the purpose of the learning other than that there is some subject matter that the teacher knows about. Here the difficulty is not congruence between learning outcomes and assessment but the learning outcomes themselves. Students simply cannot see the point of it all. They do not care enough about the course to bother with the feedback.

Programme design

I once came across feedback sheets used to structure feedback given to students that had a final section called ‘Developmental Priorities’. In effect it was saying “next time you come across an assignment like this, next semester, or next year, here are a few things you really ought to pay more attention to ……” Students were expected to tear off this section and staple it to the top of the next assignment they encountered of the same type, to remind themselves to pay attention to these things, and to inform the marker that they had already been told once before to pay attention to these things. It was a way of linking up assignments so that feedback could feed forwards, even when there was no planned coherence in programme level assessment design.

But of course it would have been much better to link up assignments in a planned way so that the level of sophistication with which students tackled essays, or reports, or whatever, followed a planned route of progression across three years of a degree programme. Students might be told “Whenever you encounter a practical report it will always be of the following form…but in the first year we will be concentrating on X, in Year 2 on Y and in year 3 on Z. The feedback we give you will be intended to help you to tackle the next practical report in a more successful and sophisticated way on subsequent courses”. Too often the focus of feedback does not progress over time, there is no programme level planned progression, and as a consequence feedback cannot feed forwards.

Employment practices

Nowadays it is increasingly common for the person who marks a students’ work and writes feedback to not be the course leader, or even to be teaching on the course, let alone be the designer of the course. Graduate teaching assistants and part time teachers are hired because marking takes a lot of time and such markers are cheap. But this comes with a price. Novice teachers are less good at their job, and particularly at marking and giving feedback. Part time teachers are seldom briefed properly about the course and its aims and rationale, and often ignore (or do not understand) criteria and do not have a good grip on standards. They do not know the students whose work they are marking and are unlikely to ever have to face them so they may care less about the consequences of their marking. The feedback of part time markers has been found to be, in general, less congruent with the course and the assessment regime as designed, than that of full time teachers. And when this happens students can find their feedback worse than useless, and very annoying. They have taken the trouble to read the assignment brief and criteria, they think they understand what they are supposed to be doing and what will get the marks, and have followed the instructions. Then they are hit in the back of the neck by misdirected feedback from markers who seem to understand none of these things.

The Open University have about 700 full time academics who design courses and their assessment systems, and about 7,000 part time tutors who do all of the marking. But they do not suffer from this problem. Their tutors are briefed properly and given detailed specification about assignments and what is intended and what they should be concentrating on. They are provided with structured frameworks for arriving at marks. They are trained in giving feedback. And their marking and feedback is monitored by full time academics. If they are not giving appropriate feedback they will be contacted and re-oriented. If their marks are out of line, ditto. If they do not respond appropriately then their marks are massaged statistically so that the tutor group’s average and distribution becomes what it should be and does not disadvantage their tutor group of students, and the errant tutors may not be re-employed the next year. The Open University has exceptionally high National Student Survey scores for assessment and feedback despite having much the highest proportion of markers who are part time. If a course is going to hire part time markers then it had better adopt some of the Open University’s practices or suffer the consequences.

Emotional climate

Students are unlikely to form a close and trusting bond with part time teachers who mark their work but who they have never met (unless, as at the Open University, the part time tutor goes to a lot of trouble to introduce themselves and establish and maintain a conversation, at a distance.). But many students have a frosty and distant relationship even with full time academics marking their work. Feedback can be experienced as cutting, insensitive and disrespectful, even if the student acknowledges the mistakes they have made in their assignment. Students’ perceptions of feedback are framed by their relationship with their marker. When the teacher has established an open and rusting relationship they can afford to be quite blunt and say things like “Come on Kim, we did this in week 2. Read the Lecture handout again” and the student will accept it and act on it. Praise is less likely to be perceived as condescending. The same feedback from a stranger may be felt to be unacceptable and may lead to disengagement, to poor course reviews or even to complaints. As with so many aspects of teaching, the nature of the emotional climate frames everything else. Open University tutors are briefed about how to establish a supportive emotional climate, to get to know their students and to adopt a personal approach, even if the only channels available are emails and an on-line forum.

Further Reading

What makes good feedback good? Applied research project run by the Assessment Standards Knowledge Exchange 

Resources for teachers and for students on making feedback more effective, from the web site of TESTA: Transforming the Experience of Students through Assessment.

Making feedback on assignments effective: principles and guidance for tutors

Making the best use of feedback on assignments: a guide for students 

This blog post is also available as a PDF

---

53 Powerful Ideas All Teachers Should Know About

SEDA is publishing online both on its website and on its blog one of Graham Gibbs’ 53 Powerful Ideas All Teachers Should Know About every week for a year, with the intention of prompting debate about the underlying basis of our work. On the blog we hope that you will comment on and discuss the ideas set out. After a year the intention is to hand over to our community and publish one idea from someone other than Graham Gibbs, each week, to continue the debate.

We invite you to join the discussion using the comment box below in response to Graham’s question.

Do your students make the maximum possible use of the feedback you provide?

If not, why not?

What might you do to make your marking and feedback efforts more cost-effective?