Introduction
Teacher education programmes in Western countries are increasingly regulated, with countries such as Australia, the USA and the UK adopting policies governing degree entry and exit, and literacy and numeracy standards of pre-service teachers. Eligibility for teacher registration is regulated through capstone teacher performance assessments (TPAs) that purport to measure ‘classroom readiness’. Teacher education is constituted as a policy problem where teacher education institutions are seen as the ‘suppliers’ of new graduates, rather than as part of a complex educational system (Mayer, 2021). This positioning has paved the way for a swathe of accountability measures and reforms under the banner of ‘improving education quality’. The discourses used to justify reform are typically built on a trust in standardisation, suggesting not only that increased regulation and accountability will improve the quality of teacher education programmes, but also that the quality of teacher education is directly linked to student achievement in the school sector. These moves represent a conflation of schooling and teacher quality, and of quality and quality assurance. Harvey (2007: 5) explains:
The processes of quality assurance are quite separate from the concept of quality. Quality is to quality assurance what intelligence is to IQ tests. Quality, in higher education is, for example, about the nature of learning. Quality assurance is about convincing others about the adequacy of that process of learning.
This article begins with a discussion of increased regulation in Australia, including the introduction of mandatory TPAs for all graduating pre-service teachers, which serve as a measure of quality of teacher education programmes and graduates. In our analysis, we are guided by the importance of understanding quality in teacher education programmes as multidimensional, rather than simply a measure used to convince ‘others about the adequacy of [the] processes of learning’ (Harvey, 2007: 5).
To locate this issue of quality, we begin with a discussion of the discourses surrounding teacher education in Australia, particularly the discursive production of teacher education programmes and educators as inherently problematic, largely devoid of quality and generally untrustworthy. We draw on Cochran-Smith et al.’s (2018) notion of intelligent professional responsibility as an appropriate framework for considering how teacher educators might adopt a more agentic stance in the face of rising surveillance and regulation. We argue for the importance of teacher educators finding opportunities to work more collaboratively across and within universities to advance professional knowledge and practice, particularly in a context characterised by increased accountabilities, including the delivery of TPAs. We then provide empirical evidence from a group of four Australian universities that have worked together to implement a nationally endorsed TPA. Examples are drawn from our data to illustrate the value of practices such as cross-institutional assessment development and moderation in expanding the professional knowledge base and expertise in teacher education. Expanding networks within teacher education also provides an opportunity to develop a broad, rich research base that might be better able to militate against further assaults on teacher education. In a context characterised by the presence of accrediting agencies seeking to exert further control in the teacher education sector, we argue for the importance of capitalising on opportunities for intelligent professional responsibility.
Distrust in teacher education programmes
Australian schooling is currently facing unprecedented pressure, including significant shortages of teachers across the country (Burke and Buchanan, 2022). This situation, along with the use of large-scale assessment data trends, has created a sense of panic and a perception of a crisis in education. The systems that enable the production of global ranking such as the Programme for International Student Assessment (PISA) have frequently featured in the Australian media and become a touchstone for policymakers, contributing to a sense of uncertainty about the state of Australian education (Crome, 2022).
One reason put forward for the purported crisis in school education is teacher preparation, based on the view that universities have failed to prepare an adequate supply of skilled teachers. In a speech announcing a 2021 review into teacher education, the then Minister for Education and Youth suggested that reform of the teacher education sector was necessary, given Australia’s declining PISA outcomes. Teacher education reform was linked to quality teaching and positioned as a mechanism for returning ‘Australia to the top group of nations’ (Tudge, 2021a: 1). The minister described the approach of educators in teacher education faculties as follows:
many teachers say they don’t feel well prepared when they enter the classroom and the number of top performing students entering teaching has declined by a third over the last 15 years, the biggest drop of any faculty. Many teacher education faculties have been infected with dogma and teaching fads, at the expense of evidence-based practices.
(Tudge, 2021b: n.p.)
Teacher education programmes (and the educators who deliver them) are thus positioned as a policy problem – accused of putting ‘the most dedicated and hard-working teachers in the world’ in the unenviable position of being underprepared for teaching due to their ‘dogma’ and lack of ‘evidence-based practices’. Such discourses are based on a distrust of teacher educators. The overarching logic is that reforming teacher education programmes will improve teacher quality, and ultimately lead to a stronger national education system. We note that these arguments obscure a multitude of structural elements impacting on teacher workforces. There are a wide range of factors implicated in teacher shortages, including increased stress, unmanageable workloads, flat career structures and a rise in insecure employment (Amitai and Van Houtte, 2022). Nevertheless, accountability paradigms use indicators such as standardised assessment and teacher attrition data as indicators of teacher education quality. The political distaste for university-based teacher preparation is grounded in a view that programmes are overly theoretical; politicians and policymakers have typically expressed a strong preference for practical skills (Cochran-Smith and Villegas, 2015).
Positioning teacher education in this way has provided justification for increased regulation of various key aspects of teacher education. The national requirement for all teacher education programmes to administer a TPA is the focus of this article.
Policy borrowing
Internationally TPAs have been used in teacher education for some time, most notably in the USA. A portfolio assessment known as the edTPA had its origins in California in the early 2000s, and it has emerged as the dominant TPA in the USA. Drawing on discourses of quality assurance, efficacy and proficiency according to professional standards, the edTPA has been adopted across the majority of states, either as part of a state-mandated programme, or by individual teacher education providers.
Amid similar rhetorical problematising moves, in 2014 the Australian federal government established the Teacher Education Ministerial Advisory Group (TEMAG) to provide recommendations on improving the quality of teacher education. The TEMAG report recommended the introduction of a ‘consistent, rigorous assessment of all teacher education students with the purpose being to ensure “classroom readiness”’ (Craven et al., 2014: 8).
The TPA policy landscape in Australia has been heavily influenced by the edTPA. It was recommended to Australia’s national accreditation body in a report that described the edTPA as an ‘authentic performance assessment … likely to be beneficial in the Australian context’ (Louden, 2015: 35). The report provided three policy options: amending accreditation standards to require a capstone authentic assessment of some kind in every teacher education programme; requiring every teacher education provider to use a specific capstone authentic performance assessment (such as the edTPA); or requiring that TPAs be nationally moderated and disclosed as part of the accreditation requirements. The Australian government instituted the first option, which allowed consortia of universities to develop and implement their own TPA programmes of assessment.
While the overarching TPA policy landscape in Australia can be seen as a form of policy borrowing (Charteris, 2019), there are also significant differences between the Australian and US contexts. Rather than insisting on one model, the Australian government decided to allow consortia of universities to develop and implement their own TPA programmes of assessment, and then to seek national endorsement. Another notable distinction is the reliance on outsourcing in the implementation of the edTPA, where edubusiness giant Pearson Education is heavily involved in implementation (Potter, 2021). This situation has led many US teacher educators to describe a feeling that they have ‘lost control over determining which students are credentialed to teach’ (Charteris, 2019: 237). However, as we will discuss, the Australian context, where teacher education providers have largely retained control over TPA design and implementation, has generated opportunities for professional collaboration amid this new form of accountability.
The burden of accountability
There is an extensive literature that documents how increased accountability mechanisms create significant burdens for those charged with enacting policy. Technologies that allow people to compare and judge themselves according to standards and statistical norms create one of the ‘gentlest and yet most pervasive forms of power in modern democracy’ (Porter, 1996: 96). Ball (2003), drawing on the writing of Lyotard, uses the notion of performativity to explain modern modes of state regulation. Performativity is ‘a mode of regulation that employs judgements, comparisons and displays as means of incentive, control, attrition and change based on rewards and sanctions (both material and symbolic)’ (Ball, 2003: 216). In accountability systems, it is typically the performances, displays of productivity and output that become a measure of quality.
Accountability mechanisms like TPAs emphasise constructs such as ‘classroom readiness’. Although there is no clear agreement or definition of classroom readiness, it is typically bound up in standardised measures of performance and data that capture evidence of teaching impact and learning improvements. In this way, ‘teacher quality and classroom readiness is calculable and reified, and the teacher’s output of educating is instrumentalised via “objective” data – but also may lead to fabrication and misrepresentation – on the basis of numbers and standardised testing’ (Rowe and Skourdoumbis, 2019: 49). Although the TPA is relatively new in Australia, literature from the USA, which adopted TPAs much earlier, points towards this trend. Au (2013) has suggested that TPAs have narrowed teacher education curricula in the USA, with the focus of teaching shifting towards TPA preparation, with an overly ‘positivist orientation’ where generating data and evidence is a dominant concern (Coloma, 2015). In so doing, intellectual rigour, deep understandings of teaching and learning, and consideration of social issues and equity are subsumed by a focus on accountability (Coloma, 2015; Greenblatt and O’Hara, 2015). In this context of rising accountability, the concept of intelligent accountability has been put forward as a way of responding to tightening accountability frameworks. O’Neill’s (2002: n.p.) exploration of these negative effects recognised that accountability often distorted ‘the proper aims of professional practice’. O’Neill’s suggestion was that for accountability systems that minimise these effects, governments needed to move away from controlling systems, in favour of good governance. To do so, she problematised the assumption that accountability is a precursor to trust – in other words, contesting the belief that new accountability regimes build trust.
Intelligent professional responsibility
The concept of intelligent professional responsibility (Cochran-Smith, 2021) has been put forward as a way of responding to measures that reduce and subordinate teacher educators’ agency in accountability processes. Intelligent professional responsibility offers an alternative, strengthened paradigm of accountability, bringing together concepts of intelligent accountability, democratic evaluation and strong internal accountability.
The first aspect of intelligent professional responsibility that Cochran-Smith (2021) draws on is the notion of intelligent accountability, as discussed above. To reiterate, the emphasis on intelligent accountability is built on trust and the active participation of stakeholders who are being held accountable, coupled with a focus on generating insights for programme improvement.
The second aspect of intelligent professional responsibility adds principles of democratic evaluation, namely inclusion, dialogue and deliberation. These principles join with intelligent accountability to generate conditions where ‘accountability mechanisms, processes, and content – i.e. what teacher education is actually accountable for – are jointly determined’ (Cochran-Smith, 2021: 14). In other words, accountability mechanisms that are underpinned by intelligent professional responsibility create spaces for teacher educators, and other stakeholders, to make genuine contributions to the design of accountability mechanisms that account for the vagaries of local settings.
The final aspect of intelligent professional responsibility emphasises how external accountability mechanisms supply opportunities to strengthen internal accountability. Rather than accountability producing orientations of compliance through obligation, strong internal accountability ‘emphasises capacity building and stakeholders collaborating within and across institutions’ (Cochran-Smith, 2021: 14). This allows teacher educators to meet external accountability demands in ways that value their professional knowledge and expertise, while generating further knowledge and innovation.
In sum, intelligent professional responsibility brings together the inevitability of external accountability with approaches based on trust, inclusion, dialogue and capacity building. This research explores the impact of an accountability measure, the quality teacher performance assessment (QTPA), on teacher educators’ work, while highlighting the underpinning logic and ideals of intelligent professional responsibility.
The quality teacher performance assessment
As described above, all Australian teacher education providers were required to have a TPA in place for their 2018 graduate cohort. In June 2019 the QTPA became the third consortium to be nationally endorsed. The QTPA is a tool developed and used by a consortium consisting of four universities from across Australia.
The development of the QTPA grew out of the desire of teacher education academics within the consortium to meet national requirements (and thus ensure national accreditation of teacher education programmes), while learning lessons from international experiences (including that of the edTPA) (Spooner-Lane et al., 2022). From the commencement of the partnership, the Executive Deans at the four universities planned for a consortium based on principles of collaboration, where each partner would contribute to QTPA implementation and development, founded on principles of fair quality assessment practice that reflected locally relevant contextual factors. One of the most significant quality assurance practices undertaken in the consortium (to ensure national endorsement expectations are met) is cross-institutional moderation. Moderation is a quality assurance process where educators come together to examine and discuss samples of student work, and to generate shared understandings of the assessment, and the standards against which the work is assessed (Bloxham, 2009). In moderation discussions, educators negotiate and confirm their judgements about students’ achievement against the assessment criteria.
The QTPA requires assessors to first work within their institution in panels of three to discuss individual students’ submissions, and, through moderation dialogues, to reach consensus on a final score. Teacher educators also moderate across the consortium, working together to reach shared agreement about individual student submissions from each university. Cross-institutional moderation, which occurs biannually across the consortium, is therefore not a ‘stand-alone’ feature of TPA design and implementation.
Evidence of pre-service teachers’ TPA results, along with outcomes of cross-institution moderation meetings, is submitted annually to the teacher regulation authority responsible for accreditation of teacher education programmes. Each teacher education provider must comment on the quality of their graduates (including percentage of satisfactory and unsatisfactory), and on where programmes could be further strengthened to support pre-service teachers’ knowledge and skills (Spooner-Lane et al., 2022).
Methods
This article presents research collected from key stakeholders within the QTPA consortium, which was comprised of four universities. Ethical clearance was obtained to conduct individual interviews and focus groups with university staff from each of the four universities that form the QTPA consortium. Sixteen participants took part in the research. Individual interviews were held with senior staff whose leadership positions related to the QTPA in their own institutions, and focus groups were held with academic staff from each university. Academic staff were involved with the QTPA as either directors or teaching staff. Interviews were conducted by a research assistant who had been a QTPA assessor at one of the member universities. The researcher had not been part of QTPA consortium meetings and was employed so that participants could speak openly about their experiences. Interviews and focus groups were conducted either face to face or via video-conferencing for a duration of approximately thirty minutes for interviews, and one hour for focus groups. Participants were asked about their views on, and experiences of, cross-institutional moderation, and how the QTPA had impacted their programmes.
Interview transcripts were analysed following Braun and Clarke’s (2006) reflexive thematic analysis method, which includes inductive and deductive processes. As they describe:
It can be used for a more deductive or more inductive analytic process (recognising this can be a continuum, rather than dichotomy). We mean inductive in the sense of analysis ‘grounded in’ the data, rather than ‘pure’ induction, because you cannot enter a theoretical vacuum when doing TA [thematic analysis] … Researchers using reflexive TA inductively need to identify, and ideally articulate in their reporting, the theoretical assumptions informing their analysis.
Reflexive thematic analysis enabled themes to be identified to capture ‘something important about the data in relation to the research question and represent[ing] some level of patterned response or meaning with the data set’ (Braun and Clarke, 2006: 82). We used an approach to grounded theory that Tucker (2016) suggests has been well established in qualitative research, and that allows for discovery of ‘anchors’ between concepts, and themes and categories that were generated from the empirical data. In our case, rather than beginning by looking for evidence of intelligent professional responsibility, we allowed categories and themes to emerge from the data, and then examined these to determine if there was evidence of intelligent professional responsibility.
The authors reviewed the data independently to generate categories and themes, before collectively confirming analysis through mutual agreement. Categories included moderation, marking and assessment design. The themes that ran across the data represented the important matters to which participants returned throughout their interviews. These included the value of collegiality across university partnerships, and the value of working across institutions using established processes – most notably, moderation – as well as the importance of a shared commitment to the development and delivery of quality teacher education programmes. These themes were representative of the data across the corpus.
As part of our reflexive analytic process, we returned to the literature on TPAs and accountability mechanisms in teacher education. We found the key aspects of intelligent professional responsibility (Cochran-Smith et al., 2018) to be evident in the data. This framework therefore became a helpful analytic tool to provide a lens as we continued to analyse the data. In particular, we found that the conditions of external accountability around the introduction of TPAs in Australia supported strong internal accountability, and that our participants’ views indicated that their focus on collegiality and quality occurred through inclusive dialogue and intelligent accountability.
Findings: ‘our preparation of our graduates has definitely been enhanced’
All of the participants suggested that taking collective responsibility for the improvement of programmes, and the tool itself, was a key feature of the consortium’s work. While TPAs are an accountability mechanism, the participants demonstrated a strong professional commitment and willingness to use QTPA processes to build rigour in their programmes. Cross-institutional moderation provided educators with the opportunity to raise legitimate concerns, and to discuss evaluation decisions through a roughly equal balance of power. One of the academics explained:
For me, the moderation process is about looking for that consistency and [understanding] … how people are making their judgements, and applying their judgements, and what people are looking for. And I think that to have a moderation team that does that, with a wider lens [across the consortium], is really important in this kind of process, particularly as we’ve got so many people involved, so many different assessors with so many different, you know, potential biases, or, you know, subjective ideas about what they’re going to be looking for, I think [because TPAs are so] high-stakes, for me, that the moderation is, you know, an absolute must, because until you’re actually able to pull that together, and have a look with a wider lens, you’ve got no idea about any of that consistency that’s happening in those individual panels.
Our participants described using moderation and consortium meetings to discuss consistency of assessment judgements and the refinement of the assessment tool to ensure that it was a meaningful measure of readiness to teach. For example, participants discussed the importance of ensuring that pre-service teachers demonstrated the capacity to differentiate teaching to accommodate the diverse learning needs of students in a wide range of contexts.
These practices align with intelligent professional responsibility, including the notion of ‘deliberative democratic evaluation’ (Cochran-Smith et al., 2018), which is built on principles of dialogue and deliberation with various stakeholders, including teacher educators. The opportunity to work across universities and programmes, openly sharing knowledge, led academics to consider how they might further strengthen their own teaching and the delivery of teacher education. Our data pointed towards numerous instances of academics reflecting on their practice and capacity building in their teacher education faculties. Participants explained how they had drawn on their QTPA experiences to refine their teacher education programmes. One academic suggested:
I guess it’s, you know, that wonderful feedback loop for educators, realising that we can always do things better, and we can always improve those things, and our preparation of our graduates has definitely enhanced, been enhanced as a result of going through this assessment process.
An academic at another university reflected on how she had worked with students to develop their beliefs about teaching and learning as part of a personal teaching statement:
I get them to create a [document early in their degree where they have to consider] ‘What does the theorist say, what does the research say, what does this mean for you?’… So it does make me think, it helps me reflect on my teaching. So if we’re thinking about this cyclical thing, we want [students] to do the TPA to reflect on their teaching practice when they’re on their final placement; it also means that we should be reflecting on our teaching throughout the course, and that’s the beauty of having all your staff part of the process.
This deep reflection on the quality of programmes happened at both the individual and the institutional levels. For example, one teacher educator described:
I’m involved at the moment with [mapping the progression of study across the primary schooling teacher education degrees]. And it has been fascinating. Just having that dialogue about, you know, what each other were doing in [each] unit, where you don’t generally have that information, and, you know, that whole mapping opportunity to go back through the four years, for example, the undergrad, and to start to look at where they’re actually developing the skills in certain areas, and valuably, where there are deficits or where you could potentially start to address some of these ... standards that students struggled a little bit with.
Similarly, another academic noted that it was not always apparent how students were developing the range of skills needed to be an effective teacher across the progression of their entire degree. They described QTPA moderation as being ‘really important’, because it enabled them to reflect on whether ‘we [are] actually explicitly exposing our students to [all of the required] standards’. They added: ‘That has been a really, real strength of this whole process as well, is ensuring that we, as a university body, can actually ensure that our students are strong in, you know, what’s required of them in the industry.’
Moderation panels were regarded as complementary to supervising teachers’ assessments of students during placement, enhancing the capacity of teacher education systems to determine the classroom readiness of pre-service teachers. This professional responsibility was something academics across all universities took seriously:
You know, you need to collect data, you need to look at students, you need to plan, which is exactly what you expect students to do in their final practicum, but now their artefact of the actual TPA itself is a much more tangible assessment task than just a ‘prac’ teacher’s report, of saying, well ‘I’ve seen your teaching, well done’, so there’s a lot more people reading and reviewing and assessing this TPA, which encapsulates and demonstrates their capacity to meet the professional standards.
Discussion
The US experience of the edTPA shows that when external providers assume significant power in the field, there is a risk of increased accountabilities that marginalise teacher educators from key decisions about teacher education and narrow the curriculum towards testing requirements. As Potter (2021) points out, although Pearson Education owns the content and distribution of the edTPA, the company name does not appear in the edTPA Handbook. Moreover, there is no publication information available about who scores the edTPA, with scorers barred from revealing their identities. Reagan et al. (2016) explain that Pearson Education not only determines who scores the edTPA, but also provides assessor training, and data collection and reporting. In contrast, the Australian context has provided teacher educators with a unique opportunity to retain control over important aspects of the process.
These external conditions are somewhat unusual, because they provide teacher educators with the opportunity to come together and to play a role – to some extent, at least – in determining how accountability mechanisms operate. In doing so, they demonstrate the collective capacity of educators to exercise professional agency under conditions of reform (Rayner and Gunter, 2020). This provides the sector with an opportunity to test the ‘critical unspoken assumption that … unless teacher education programs are coerced and surveilled by external agents, they are unlikely to perform appropriately, adequately or effectively’ (Cochran-Smith, 2021: 18). Our data demonstrate that, in this context, academics in teacher education are able to take responsibility for assuring quality to external bodies while using the process to reflect on the quality of their own programmes and teaching.
Unlike external accountability structures which can lead to performative activity, Australian universities are in a position where they have had the opportunity to draw on their own expertise to develop robust accountability mechanisms that may enhance teacher education programmes. In the QTPA, regular internal and cross-institutional moderation processes were seen by participants as a means of ensuring consistent quality practices across the teaching and learning cycle, and national consistency in how to best prepare and assess pre-service teachers’ classroom readiness. Participants indicated that shared discussions provided learning opportunities that aided teaching and curriculum development. Including teacher educators in the TPA assessment process led to reflection, where they reported gaining a deeper understanding of the classroom readiness of graduating students. The external environment, which led to the creation of consortia, meant that discussions spanned institutions and state borders, creating a unique opportunity to build shared knowledge and understanding about teacher education programmes, and graduate readiness to teach. We suggest that the current policy environment has created new opportunities for the development of strong professional relationships across institutions, with national opportunities to build teacher educator expertise and experience.
Conclusion
In her presidential address at the Australian Association for Research in Education in 2011, Reid (2011: 388) argued that ‘when standards “stand in” for knowledge about teaching, and when teacher education graduates are required to demonstrate a capacity to teach “in the language of the standards”, they are demonstrating their capacity to “join the dots”’. In this article, we have used the example of one Australian TPA, the QTPA, to illustrate how teacher educators might demonstrate intelligent professional responsibility in their commitment to delivering quality teacher education programmes. This is especially important given the US experience, in which Pearson Education’s involvement has diminished the power of teacher education academics in how programmes are constructed and delivered. We contend that a further takeaway is the importance of Australian teacher education providers advocating for the importance of their work, and the importance of avoiding relinquishing control over curriculum and assessment decisions, should future teacher education reviews raise the possibility of TPAs being outsourced.
Through the example of the QTPA, we have argued for increased intelligent professional responsibility and democratic evaluation, such that teacher educators can have a say in what counts as quality teacher preparation. Our data illustrate the value of teacher educators reflecting on and developing their teaching. The implementation of TPAs within university consortia has shown that intelligent accountability along with intelligent professional responsibility is possible, and that it can generate new opportunities to strengthen teacher education programmes by sharing knowledge and expertise.