You are on page 1of 42

Examining washback

:
What do we know
and what is there left to explore?

Dianne Wall
Trinity College London

Definitions
High-stakes tests

Tests whose results are seen – rightly or
wrongly – by students, teachers,
administrators, parents or the general
public, as being used to make important
decisions that immediately and directly
affect them.

(Madaus, 1988)

Definitions
Impact
Any of the effects that a high-stakes test
may have on individuals, policies, or
practices – within a classroom, a school,
an educational system or society as a
whole.

(Wall 1997)

Definitions
Washback

Washback is a type of impact, which 
relates to the effects of high‐stakes 
tests on classroom practices –
particularly teaching and learning.

to the extent that it either promotes or impedes the accomplishment of educational goals held by learners and/or programme personnel. (Bailey. 1996) . Types of washback … can be positive or negative.

p. Positive washback If a test has positive washback. ‘there is no difference between teaching the curriculum and teaching to the test’. 1997. (Weigle & Jensen. 205) .

may lead to the abandonment of instructional goals in favour of test preparation . Negative washback ‐ a mismatch between the stated goals of instruction and the focus of assessment. .

Outline • Development of the concept • Overview of research early studies overview of what we know now • Critical issues .

low-cost means of influencing the quality of what teachers teach and what learners learn at school. General education Positive views Tests can be a powerful. (Heyneman & Ransom. 1992) .

1987) . then focusing teaching on what they assess is a beneficial activity. (Popham. General education Positive views ‘measurement-driven instruction’ (MDI) If tests are ‘properly conceived and implemented’.

1989) . General education Positive views Systemic validity …when a test induces in the education system curricular and instructional changes that foster the development of the cognitive skills that the test is designed to measure. (Fredericksen & Collins.

1988) . demeans the professional judgement of teachers. and finally. General education Negative views ‘psychometric imperialism’ ‘… invariably leads to cramming. (Madaus. narrows the curriculum. concentrates attention on those skills most amenable to testing…constrains the creativity and spontaneity of teachers and students.

 1988) .… transfers control over the curriculum  to the agency which sets or controls  the exam. (Madaus.

then it will have a damaging effect on teaching. (Heaton. if bad. Views from language education If it is a good examination. 1990) -Aspirations -Assertions (little evidence) . it will have a useful effect on teaching.

Foundation papers • Alderson & Wall 1993 • Hughes 1994 • Bailey 1996 • Messick 1996 .

Alderson & Wall 1993 • What washback is and what it is not • Need to be specific: Washback Hypotheses • Need to use a variety of methods when investigating – with emphasis on observation • Need to learn from research in other disciplines: e.g innovation theory. motivation theory .

• A test will influence what teachers teach. • A test will influence what learners learn. • A test will influence learning. Washback Hypotheses • A test will influence teaching. • A test will influence how teachers teach. . • A test will influence how learners learn.

Washback Hypotheses (continued) • A test will influence the rate and sequence. or • Tests will have washback on some teachers and some learners but not on others. • A test will influence attitudes to the content. etc … of teaching and learning. • Tests will have washback on all teachers and learners. method. . • A test will influence the rate and sequence. and the degree and depth of learning. and the degree and depth of teaching.

Hughes 1994 Need to look at washback on -Participants -Processes -Products .

Participants Teachers Learners Administrators Materials writers Curriculum designers .

how they teach. deciding what to study and what not to study. memorising. whether they give extra classes… For learners Practising the target language and skills. looking for shortcuts. cheating… . Processes For teachers What they teach. worrying. the intensity of their teaching.

Products New curricula? New timetables? New materials? Better learning? .

Bailey. 1999 Basic model of washback .

Messick 1996 Evidential link It is problematic to claim evidence of test washback if a logical or evidential link cannot be forged between the teaching or learning outcomes and the test properties thought to influence them. .

seek validity by design as a likely basis for washback. . Messick 1996 Validity by design …rather than seeking washback as a sign of test validity.

tasks) •avoid construct under-representation •avoid construct-irrelevant variance . Messick 1996 Validity by design •direct testing •authenticity (input.

 means of assessment Alderson & Hamp‐Lyons 1996. Watanabe 1996 teachers are affected by tests in different  ways Shohamy. 1996 the washback of tests can change over time . Early research Wall & Alderson 1993 different amounts of washback – on content. Donitsa‐Schmitt & Ferman.  methods.

teaching of test-taking skills Attitudes and feelings of learners and teachers Learning Do test results improve? Does learning improve? . teacher-made materials Teaching methods choice of methods. use of past papers. Areas where washback can appear (Spratt 2005) Curriculum contents of curriculum. timetabling Teaching materials choice of textbooks.

Factors influencing washback (Spratt 2005) The exam -its proximity -its stakes -the status of the language it tests -its purpose -the formats it employs -the weighting of individual papers -when the exam was introduced -how familiar the exam is to teachers .

Teacher beliefs about ‐the reliability and fairness of the exam ‐what constitute effective teaching methods ‐how much the exam contravenes their current  teaching practices ‐the stakes and usefulness of the exam ‐their teaching philosophy ‐the relationship between the exam and the  textbook ‐their students’ beliefs .

Teachers’ attitudes towards ‐The exam ‐Preparation of materials for exam classes ‐Lesson preparation for exam classes Teachers education and training ‐Teachers’ own education and educational experience ‐The amount of general methodological training they  have received ‐Training in teaching towards specific exams and in how  to use exam‐related textbooks ‐Access to and familiarity with exam support materials  such as exam specifications ‐Understanding of the exam’s rationale or philosophy .

Resources ‐The availability of customised materials and exam  support materials such as exam specifications ‐The types of textbooks available School ‐Its atmosphere ‐How much the administrators put pressure on  teachers to achieve results ‐The amount of time and number of students in  exam classes ‐Cultural factors such as learning traditions .

& Pitkänen-Huhta 2006 Tracing the evidential link – Wall & Horak 2006. 2008 and 2011 . Recent investigations Teacher beliefs – Huang 2009 Washback on learners – Huhta. Kalaja.

Critical issues • For teachers ‐ How to find the right balance  between teaching and exam preparation? • For exam developers – How to strengthen the  influence of the exam when so many other  factors affect teaching? • For testing community ‐ Who should accept  responsibility if an exam is mis‐used. and has  unintentional negative washback and impact? • For educational authorities – How to convince  examination boards to do impact studies? .

Is the test intended to initiate change(s) in the current practice? 2. What is the washback effect? What studies have been conducted? . Importance of washback studies EALTA Guidelines for Good Practice Considerations for test development in national or institutional testing units or centres 1.

A prova tem como finalidade iniciar alterações  na prática actual? 2. Os professores recebem formação para  preparar os estudantes para a prova ou exame? . Directrizes EALTA para a Boa Prática  na Avaliação de Linguas 1. Foram realizados estudos para determinar as  repercussões da prova no ensino? 3. Existem materiais de apoio para os estudantes  que pretendam realizar a prova? 4.

 when. how? . what. Survey on Impact Studies (Wall & Horak 2006) Survey of EALTA members via discussion list – Have you been involved in the introduction of a new  exam or an examination reform? – What was the intended impact of the new/reformed  exam? – Was an impact study of any sort conducted (however  small)? – Methodology – who.

Survey on Impact Studies Reasons given for not conducting an impact study • Lack of awareness of value  ‘The new system is simply assumed to be a  substantial improvement …’ • Lack of interest ‘…we can’t imagine who would have been  interested in the results except perhaps  ourselves.’ .

’ . we did not have the time or energy to  conduct any further tests of the kind you mentioned.’ • Lack of time and energy ‘We knew it would be hard enough to introduce  reforms. Reasons given for not conducting impact studies • Lack of expertise ‘…because there is no expertise available and  there is not a scientific language testing tradition in  the region.

Reasons given for not conducting impact studies • Lack of resources • Fear of results ‘Ministries are notoriously reluctant to let  outsiders see if their measures have measurable  outcomes.’ .’ ‘…fear to stir the hornet’s nest – the reality might  be too terrible.

Why aren’t there more washback studies? Lack of  • Awareness of the value  • Interest • Expertise • Time and energy • Resources and • Fear of results .

 and over to you… Dianne. Thanks.uk .co.Wall@trinitycollege.