Assessing Children’s English – Again

If I asked teachers to ‘agree or disagree’ with the following statements, what would most say?:

‘The purpose of assessment is to allow students to demonstrate what they know’.
‘One function of assessment is to increase student motivation to learn’.
‘Assessment should give students a sense of achievement’.
‘Assessment should encourage creativity’.

My hunch is that most teachers would agree with such sentiments. Translating them into practice, though, remains a challenge and the way students’ English is assessed is very often in stark contrast to such ideas. I reflected on this in an earlier blog called ‘Why do teachers assess English the way they do? A recent test paper again highlights important issues in the assessment of children’s English and I would like to comment further.

I will take some examples from a recent English test for 13-year olds in Slovenia (though I am sure the issues highlighted apply more widely). Overall, about 90% of the test involved grammar-focused gap-filling work, with some vocabulary and translation exercises. Here is one example:

The question tested students’ knowledge of phrases following the pattern ‘That looks/ sounds ….’. In this case the student provided three reasonable answers, two of which were awarded zero marks because of spelling errors. There is no recognition of the students’ knowledge of the target expressions and the focus on spelling implies that accuracy is all that matters.

The test also contained a gap-filling vocabulary exercise. Here is an extract:

This looks like a free exercise but students were in fact expected to remember words used in their textbook. So, while ‘went to the USA’ makes reasonable sense and might deserve half a mark, ‘emigrated’ was the expected answer. And, once again, spelling errors are heavily penalised, with zero marks awarded for ‘vulcanic eruption’. I am sure that many creative answers to the item about Scotland were possible – but ‘parliament’ (spelled correctly) was I suspect the only one that would have been marked correct.

Some of the translation items (Slovene to English) were also interesting. One was this:

Question: Vojska Edwarda I je v 13 stoletju osvojila Wales.
Translation: The army of Edward I conquered Wales in the 13th century.

The rationale for such content is that it appears in the textbook and students should therefore know it. But there must be more relevant and meaningful language contexts in which to use translation as an assessment tool. In the example I reviewed, the student scored zero marks on three items of this kind. Overall, they received the lowest possible grade and it is unlikely that the experience will have stimulated in any way their interest in learning English.

How teachers assess is often largely a matter of tradition – things are done the way they always have been and new teachers entering the system are quite quickly assimilated (see my earlier blog on assessment in pre-service teacher education). Changing traditions of this kind is only possible with education, informed leadership , collective engagement by teachers and, importantly, greater overall awareness of the impact (positive and negative) that assessment can have on children’s attitudes to learning English.

Much information is available online to support the development of teachers’ assessment literacy. I would once again recommend The British Council’s Teacher Assessment Literacy project, including this video on assessing young learners. This Handbook of Assessment for Language Teachers is also an excellent resource.

If you are involved in supporting the assessment literacy of pre-service or in-service teachers of English, please consider using the above examples (and those I discussed earlier) and relevant online resources to encourage teachers to reflect on their assessment practices and, very importantly, the impact these have on students’ attitudes to English.

Posted in professional development, teacher education | Tagged , , , , , | Comments Off on Assessing Children’s English – Again

COVID and Language Teacher Education: New Research

Discussions of the impact of COVID–19 on education are understandably often framed in negative terms; the pandemic has, after all, had a massive disruptive impact on schooling globally. Teacher education – both initial teacher preparation and continuing professional development – has also been adversely affected by the often enforced transition from face–to–face to online modes of learning. A new study examines the impacts of the shift to online teacher education in the field of English language teacher education. Through a series of case studies, it highlights the challenges that teacher educators and trainees experienced, how teacher education changed in response to these challenges and the factors that facilitated such change. The full report will be available in the next few days at www.aqueduto.com/research/.

The study also highlights, though, many positive outcomes for teacher education programmes of the enforced shift to online delivery and in this blog I would like to focus on these. They illustrate how the educational crisis created by COVID–19 was a stimulus for productive change that would have otherwise taken longer to occur or not happened at all.

Nine providers of English language teacher education from around the world – all largely new to online delivery –  took part in the study. Through detailed interviews, they told their story of how COVID–19 impacted on their work, including, despite many challenges, several ways in which the experience had ultimately been beneficial for them. The box below summarises the benefits they highlighted and these are discussed further in the report.

The enforced move to online teacher education led to improvements in ….

Understandingof the value of online education; of the differences between online and face–to–face education
Awarenessof the challenges trainees face in an online environment
Competenceamong teacher educators and trainees regarding online tools and platforms and of how to work effectively online
Collaborationas teacher educators worked together (and with trainees) to find effective online solutions
Opportunitiesto expand business to new online markets
Efficiencyby reducing the logistical demands and costs of face–to–face teaching and learning
Accessto teacher education by a wider range of trainees
Resourcesas teacher educators located, adapted and designed digital materials
Flexibilityin the ways teacher education was delivered and (where relevant) assessed
(trainee refers to both student teachers and in-service teachers)

Many of these benefits (such as collaboration and increased attention to trainees’ needs) have the potential to continue to enhance the work of teacher educators even when they return to face–to–face delivery. Others will continue to shape the teacher education provision of those who sustain an online presence alongside or blended with in–person delivery.

If you are a teacher educator who was thrust by COVID–19 into a rapid shift to online delivery, are you able to identify any benefits of the experience and which might extend the list above?

Report details:

Borg, S. (2022). COVID-19 and the shift to online language teacher education. Available from www.aqueduto.com/research/

Posted in professional development, research, teacher education | Tagged , , , , , , | 1 Comment

Video-Based Observation on Teacher Development Projects

One of the challenges on large–scale teacher development projects is observing enough lessons to arrive at some general conclusions about teachers’ classroom practices, before, during and after an intervention. Various factors related to budget, human resources, and geography do in fact mean that project evaluations often involve very modest numbers of observations, relying more heavily on self–reported data through, for example, questionnaires. COVID, by limiting travel, has made collecting observational data from classrooms even harder. One assignment I worked on for the British Council recently explored how video might be used to address such issues and facilitate the collection and analysis of a larger number of lessons on teacher development projects.

The first part of the work involved a literature review. The initial aim was to review material where video had been used to evaluate teacher development projects, but it soon became clear that very limited work of that kind was available. So I expanded my searches beyond education, including material from other fields, for example, medicine and engineering. Also, while it was clear that video had been widely used (a) as part of observational research and (b) to support teacher professional development, across disciplines there was little evidence of its use to evaluate changes in behaviour following a development project.

In the end 25 recent papers were identified, and these were analysed for insights into the benefits of using video to evaluate behavioural change, the challenges of using it for this purpose, and ways of using it more effectively to study how interventions impact on what people do. As a result of the review, a number of recommendations were also made for the effective use of video in evaluating behavioural change. If you are interested in reading more about these issues, the paper is available open access at https://doi.org/10.1016/j.evalprogplan.2021.102007.

Posted in Uncategorized | 2 Comments

Incongruence in Pre-Service Teacher Education

“Teacher educators seem to agree that, to be able to support their student teachers’ learning, they themselves should be good models of the kind of teaching they are trying to promote. However, it is clear from the literature that this congruent teaching is not self‐evident in teacher education”. (Swennen, Lunenberg, & Korthagen, 2008, p.531

Two kinds of congruence are important in teacher education. The first is congruence between theory and practice – i.e. between what student teachers are taught and what teacher educators do on their courses. The second is congruence between modelled and target practices – i.e. between what teacher educators do and what student teachers will be expected to do in their own classrooms. Both these forms of congruence are linked (see Figure 1): as far as possible, the effective educational practices that student teachers learn about will be enacted in the work of their teacher educators who, at the same time, are providing a model of appropriate future professional practice for student teachers.

Figure 1: Congruence in teacher education

It is often the case, however, that a concern for congruence does not characterize the design of either whole pre-service teacher education programmes or individual courses within them. This can be illustrated with particular reference to assessment. On one programme I know of, student teachers study principles for effective assessment (see p2 here for an example of a useful checklist for assessment in HE) but the manner in which assessment takes place often seems to be at odds with such principles. So, while, for example, they learn about the importance to assessment of clear objectives, reliability, validity and feedback:

  • student teachers receive limited prior information about assessment formats (such as the nature of a forthcoming examination)
  • it is unclear to student teachers what they need to study prior to an examination (scope of the assessment)
  • student teachers are assessed against skills they have not been given opportunities to develop
  • content not focused on during a course appears in an examination
  • the scope of the content to be assessed is unfeasibly broad
  • assessment emphasizes the recall of facts
  • assessment rubrics are vague
  • student teachers receive no or limited written feedback to explain assessment outcomes
  • multiple pieces of work contribute to a final score but the breakdown of marks in the result is not explained
  • assessment criteria are not pre–defined or referenced when results are communicated
  • assessment is entirely summative
  • different teacher educators (even those co-teaching a course) assess in very different ways.

Collectively, such practices create strong tensions among the theory of assessment that student teachers are taught, what teacher educators do, and the assessment practices future teachers are expected to adopt. These tensions can have a negative impact on student teachers; for example, they may come to believe that what they study on their programme and what happens in actual educational practice are not connected. It is thus important to minimise such tensions (not only in relation to assessment but also for teaching and learning more generally) and this can be achieved through a process of reflection and alignment in which teacher educators ask themselves (and – given the value of collaborative reflection (pdf download) – ideally one another) questions such as these:

  • what educational principles regarding teaching, learning and assessment are the student teachers studying (on this programme or course)?
  • to what extent are my teaching and assessment practices as a teacher educator aligned with such principles?
  • what factors might explain any lack of alignment?
  • what changes in my approach would strengthen the congruence between what the student teachers are taught, what I do, and what we would like them to do as future teachers?

Posted in teacher education, Uncategorized | Tagged , , , , | 6 Comments

Remote Teacher Education

COVID–19 has made remote teaching a necessity for educators around the world. Levels of preparedness for this sudden transition have varied, with the result that many educators have faced the challenge of adapting rapidly and learning by doing, while continuing to support their students as best as they can. One of my current projects focuses particularly on teacher educators and the competences they need to support teachers remotely.

While much has been written about the digital competences that teachers and students need, less work is available on the knowledge and skills that remote teacher educators require. Some sources I have found useful are:

One recurrent point from this literature is that working online is different, in terms of both educator roles and the skills they need. Another is that the direct transfer of face–to–face teaching strategies will not be effective in an online environment.

While frameworks such as TPACK and CoI were designed with teachers in mind, they are helpful in thinking about what online teacher educators need to know and be able to do. From TPACK, we can conclude that teacher educators will need a sound general knowledge of and familiarity with technology, a more specific understanding of the technologies that can support teacher learning and a knowledge of how to use these technologies pedagogically in a teacher education context. According to CoI, effective online learning environments are characterised by social presence, teaching presence and cognitive presence. Online teacher educators, therefore, need to be able to create supportive social conditions, appropriately designed and facilitated teaching, and concrete, contextualised and deep learning.

In the project I have been working on (which was supported by the British Council) , we were particularly interested in in–service language teacher educators working in less well–resourced contexts. Over 400 respondents from 71 countries completed an online survey and told us about their experiences as remote teacher educators. Some key findings were that:

  • almost half of the respondents had their first experience as remote teacher educators in 2020
  • smart phones and mobile internet connections (rather than laptops and tablets) were the technologies that were most widely used to support teachers remotely (although the cost of mobile data was often a concern)
  • common remote teacher education practices among the respondents were online video meetings, live online workshops and social media group discussions
  • WhatsApp, video conferencing apps and e-mail were the tools teacher educators used most commonly to share content with teachers (see Figure 1).

Figure 1: Sharing content remotely with teachers (N=426)

When asked about the competences they needed to work remotely, the three most common answers provided by the teacher educators were ‘creating a positive online learning environment’ (89.2%), motivating teachers to be active in online groups (80.8%) and helping teachers improve their own digital competences (79.1%).

Reflecting on their experiences in 2020, these remote teacher educators were quite positive; most felt that working online had been challenging but also enjoyable. They also felt that they had access to the support they needed for this new role. One general challenge, though, was that among the teachers they worked with remotely, levels of digital competence were often modest.

While an interesting start at developing a better understanding of remote language teacher education practices and the competences that remote teacher educators need, surveys of this kind (and this other relevant report) are limited into the insight they can provide into important questions about processes, quality and outcomes. More work is needed, then, to examine

  • what exactly remote language teacher educators do
  • the factors that shape their pedagogical approach as remote teacher educators
  • the quality of remote teacher educator practices
  • what teachers learn through remote teacher education
  • the kinds of support remote teacher educators need.

Posted in professional development, teacher education | Tagged , , , | 2 Comments