Edited by David Baume and Celia Popovic
Routledge – The Staff and Educational Development Series
Publication January 2016
Chapter summaries and extracts will on the SEDA Blog over the coming months. (There may be small differences between these and the published versions)
Please add your comments!
Chapter 9 – Quality Assurance and Quality Enhancement
This chapter explores what characterises approaches to quality assurance and quality enhancement by external quality agencies, looking at definitions and at practices associated with each. It explores why Quality Assurance and Quality Enhancement are often seen in oppositional terms, both philosophically and in the associated practices. It discusses how QE, with its stronger future and improvement focus, can provide a positive context for academic developers to encourage engagement with student learning and pedagogic practices. It explores the productive alignment of assurance with enhancement in mature quality systems in the creation of internal quality cultures centred on learning, development and enhancement.
Over the last decade, quality assurance across Europe has moved towards more enhancement-oriented approaches (Kastelliz et al., 2014). This reflects factors including the growing maturity of institutions in their internal quality assurance and, following one or more initial cycles of external review, greater trust in both institutions and external quality processes.
Quality is not only about assurance or enhancement; it also involves systems, management and culture. The European University Association (EUA) define a quality culture as combining ‘shared values, beliefs, expectations and commitments towards quality’ and ‘a structural/managerial element with defined processes that enhance quality and aim at coordinating efforts’ (Sursock, 2011, p. 9, quoting EUA, 2006).
When both internal and external quality systems are enhancement-led, both are working towards excellence and to ensuring the best student learning experience, perhaps even the best student learning – more on this in Chapter 10 on outcomes and evaluation. The goal is excellence within available resources. External quality enhancement works with an institution’s mission, goals and values. Internal quality enhancement is marked by initiatives to improve educational quality at all levels, from the strategic to the individual, and will enable both the achievement of larger strategic goals and the setting and meeting of more local priorities, for example being responsive to discipline variations.
Quality enhancement in Scotland represents a well-established and mature enhancement-led approach. The approach to quality in Scotland was designed as Quality assurance and quality enhancement an integrated Quality Enhancement Framework (QEF). The QEF was developed in partnership by the key players – funding body, quality assurance agency, the university sector and students’ representative body. Introduced in 2003, it is in its third cycle. It has retained strong support, partly because the approach demonstrably works (Scottish Funding Council, 2010, and Dempster et al., 2014) and also because it lives by its own values:
- Evidence-based external evaluation and review,
- Collaborative practices and partnership,
- Openness and
Scotland’s integrated framework has five elements:
1 Enhancement-Led Institutional Review (ELIR),
2 Institution-led quality review,
3 Public information,
4 Student engagement in quality, and
5 The programme of national Enhancement Themes (‘Enhancement Themes’, n.d.).
Consensus around the QEF was achieved because there was already a climate of trust, and because the key agencies involved in its creation worked in partnership with a common aim toward common values. The QEF was jointly developed and agreed in 2003 by the Scottish Funding Council, the funding body; Universities Scotland, the representative body of the universities; QAA Scotland, the quality assurance agency; and National Union of Students Scotland, the students’ representative body. Different areas of responsibility were clear and the interrelationship between them well understood, including the assumption that enhancement includes assurance. The QEF embraces internal responsibilities, external elements and collaborative aspects, with accountability and public reporting.
Chapter 10 – ‘Is it working?’ Outcomes, monitoring and evaluation
Lorraine Stefani and David Baume
This chapter works with three principal ideas:
1 It is very important to establish, at a very early stage in planning, and as clearly as possible, what any academic development venture is intended to achieve, what its intended outcomes are. These may, by negotiation, be changed; but they should always be clear.
2 These intended outcomes provide a secure basis for monitoring (on the way through) and evaluating (at or after the end of) the venture, and thereby both for guiding the venture to success and for learning to inform future development ventures.
3 These intended outcomes should be increasingly ambitious, as academic development takes a leading role in changing higher education. The chapter seeks to justify and extend these ideas, and to provide practical guidance on implementing them, in the complex and sometimes messy world of higher education.
On terminology – Evaluation is used here to mean making evidence-informed judgments about the value or effectiveness of an educational process.
The primary intention of this chapter is to support academic developers to plan, monitor and evaluate their work in a range of ways, in order to provide accurate and usable evidence of effectiveness and added value.
To evidence our value, we developers need to ask difficult questions about what it is that we are evaluating (e.g. Stefani, 2010; Cousin, 2009) as a start to taking a more consistent and scholarly approach to investigating our own work and to creating an improved discourse of evaluation.
We suggest starting with a clear articulation; preferably derived from negotiation with stakeholder groups; of the intended outcomes of any academic development endeavour. The activities that developers undertake and the outputs that we produce are mainly important insofar as they contribute to the outcomes to be achieved. This process of identifying intended outcomes is illustrated later through two hypothetical case studies. Golding (2013) talks eloquently of the loss, the restrictions, which result from defining ourselves by what we do rather than by what we achieve.
There are perhaps four main reasons to monitor and evaluate. A good evaluation will address all four purposes:
The report should show that the resources allocated to the project were properly expended…
As well as showing whether and to what extent a project achieved its intended outcomes, a good evaluation also offers some explanation of why…
3 Contributing to improvement
This follows directly from the previous point. Monitoring should be a frequent, indeed as close as possible to a continuing, feature of the operation of any academic development venture…
4 Enhancing capacity for evaluation
… we may wish to help our colleagues … to develop some further enthusiasm for and capability in evaluation.
We suggest that courageous leadership is required of academic developers in promoting, supporting and evidencing ambitious change goals, and then establishing a robust link between academic development interventions and enhanced organisational performance, perhaps moving towards identifying the Return on Investment referred to earlier in this chapter.
Schroeder and associates (2011) provide a road map for academic developers for the future. They affirm the mission-critical role for faculty developers in this time of immense flux and uncertainty for universities. Along with other commentators (e.g. Gordon, 2010), they emphasise that the broad-based institution level changes that are underway involve many players, not only academic and educational developers.
Chapter 11 – Working with networks, microcultures and communities
Katarina Mårtensson and Torgny Roxå
‘I have really become inspired to do new things in my teaching through this pedagogical course, but I cannot do it. My colleagues are like a wet blanket over everything’. (Participant in a pedagogical course, 2003)
This chapter draws on a sociocultural perspective on academic development. It briefly introduces some current theoretical frameworks, including social network theory, organisational culture and communities of practice. It shows how these models can be used to design a variety of academic development activities. The purpose of using this perspective is to help academic developers support not only individual teachers but also the collegial context in which the teachers are professionally active. Case studies and quotations from academics are included.
Academic teachers do not act in isolation. On the contrary, they form their understanding of teaching and learning, and conduct their teaching practice, in relation to a context that includes their disciplinary community, students, academic colleagues, leaders, the organisation and society. For decades, academic developers have organised formal learning opportunities for academic teachers through workshops, courses and programmes. Academic developers have engaged in various development projects, and lately also worked strategically with leaders and managers (Gibbs, 2013). Evaluations of such activities, in search for evidence of meaningful and substantial impact on academic teaching and student learning, repeatedly indicate that the impact is not straightforward (Chalmers et al., 2012; Gibbs and Coffey, 2004; Prosser et al., 2006). This is explored further in Chapter 10 of this book, on monitoring and evaluation. The evaluation results are generally positive in terms of satisfied participants, who adopt a more student-/learning-centred approach (Prosser and Trigwell, 1999) to their teaching after they have completed a staff development programme.
However, less is known about the effects on an organisational level (Trigwell, 2012). In fact, evaluation studies hitherto indicate that the effects differ depending on how the academics’ collegial contexts value teaching and academic development programmes, as illustrated by the quote at the head of this chapter. Informal structures in the various contexts where academic teachers are active thus appear to mediate the impact of formal academic development activities.
Van Maanen (2007; further developed in Ancona et al., 2009); helpfully defines three different lenses with which we can analyse and understand an organisation:
1 The strategic design lens, which reveals how an organisation is intended to work, often illustrated with charts of boxes and arrows on an organisation’s website;
2 The political lens, through which we can see different interests, stakeholders, alliances and power struggles within an organisation; and
3 The cultural lens, through which we can see the norms, habits, traditions and acts of meaning-making constructed through day-to-day interactions and activities in the organisation.
We claim that it is mainly through day-to-day interactions and meaning-making processes in collegial contexts that academics develop their understanding and practices of teaching. We have argued that academic developers may benefit from taking these collegial contexts into account when designing activities and support for the development of teaching and learning within an institution. We also have provided examples from our own university of what academic developers can do. The chapter thus outlines how academic developers can work in several different ways to support, not only individual academics, but also their collegial contexts in which their teaching is practiced. By designing academic development with the specific purpose of supporting both individuals and their local context, their microculture, we think academic development initiatives can reach a greater potential in their endeavour to improve student learning.
About the Editors
David Baume is an independent international higher education researcher, evaluator, consultant, staff and educational developer and writer.
Celia Popovic is Director of Teaching Commons at York University, Toronto, Canada.