From the Surrey Association of Teachers of Modern Languages (November 2007)
We wish to express our concerns around the newly implemented oral e-exams
for second language provincial exams. Our preference would be to discuss
these concerns with the creators of the concept for these exams, but it
seems that
any discussions, requests for discussion and informed professional debate
before implementation are denied and actively discouraged.
It is important to contextualize the concerns raised in this letter with
the following significant facts.
Firstly, second language teachers have asked for and do want an oral component in the provincial exam. Oral skills are vital in learning a second language and should be tested as well.
Secondly, evaluation and assessment should reflect best classroom practice. On the other hand, best assessment practices can also encourage teachers to use more effective methods in teaching and, as a result, increase student achievement. The proposed technique/method for testing speaking in particular does not reflect best classroom practice nor does it encourage the same.
Thirdly, the second language Learning Outcomes and teaching methodology stress interactive oral communication (spontaneous interaction, discussion, debate, exchange of ideas, etc.) Where the outcome is reporting or narrating (which the exam “monologue” seems to most closely resemble), the illustrative examples in the IRP Enhancement Package refer to “prepared” speech. The proposed oral exam tasks do not reflect the communication emphasis in the IRP.
We have organized our concerns into different categories for greater clarity.
We have offered suggestions for an improved process.
Pedagogical issues
Pedagogical issues constitute our primary concerns and need to be front and
centre when discussing the e-exams.
• Learning Outcome intentions, interpretations and compliance
As stated above, the second language Learning Outcomes and teaching methodology
in the BC IRPs stress interactive oral communication. Where the outcome is
more akin to a monologue, the illustrative examples (in the IRP) refer to “prepared” speech
that is usually based on some research, reading, viewing or experience. It is
questionable whether the intent of the LO of narrating or reporting is actually
captured in the proposed “monologue” task. If not, which LO are
we in fact addressing?
The short answer (general) questions, which are a series of unconnected and
unrelated questions, do not resemble or simulate natural conversation patterns
either.
They are more like an interrogation that solicits isolated responses which
is anathema to best communicative practice. It looks like one small oral strand
now comprises a large portion of a student’s final mark while the more
relevant interactive communication outcomes are disregarded.
• Authenticity of assessment tool
Authenticity and reflection of real-life situations is something that L2 curricula
stress and teachers strive to bring into the classroom. Teachers strongly object
to summatively assessing oral skills by the narrow method proposed. An individual
talking into a microphone without a defined audience is not as authentic a
task as one would want. In real life, even students who use technology a lot
when “communicating” do
so in a meaningful way with a person or an intended recipient. We have better,
proven methods of assessing that mirror the LO intentions and real-life situations
more appropriately.
• Best classroom practices
Our best classroom practices have a strong oral strand. In fact, oral communication
is at the heart of language teaching and students are able to sustain conversations
and discussions of considerable length when they have the appropriate teaching
and practice activities. Our most successful results come from following the
suggested procedures and rubrics found in the on-line Ministry document “Assessing
Speaking Skills in Core French”. This methodology of assessing speaking
skills has also been very effectively used in other exam situations for second
languages (e.g., the Lower Mainland Language Challenge exams). The process is
entirely interactive (student with student); it builds and elicits conversational
gambits, mutual understanding, sophisticated persuasive speech etc. The new e-exam
is a big step backwards in shaping oral competence. Teachers will now feel obliged
to practise and drill more unauthentic and isolated response patterns to the
detriment of developing better real communication skills.
Furthermore, this type
of task lends itself to memorization of “speeches” on the given
topics, which is definitely not what the LOs encourage.
• Some questions teachers have:
1. Have the rubrics used for marking been established, widely piloted and researched
with ALL languages in advance?
2. With the addition of these new components to the provincial exam will the
expectations in level of difficulty be scaled down for the other components
to reflect this extra need for oral practice?
3. What kind of training will be available so that teachers are clear on the
process as well as the criteria and mark consistently?
Administration of exam
• Support-needy exam process
The proposed process has unnecessary exigencies. Not only does the teacher have
to be fully involved in the preparation and administration, but school and
sometimes district IT personnel will be essential. In addition, substantial technology
help is required at the Ministry level.
Lab time will have to be booked for ALL language classes for practice in advance
as well, adding complexity and strain to a system that is already overwhelmed.
Embedded in these requirements are logistical problems that “more money
and technology” disbursed to the schools won’t solve either. One
excellent teacher, for example, was not able to administer the exam to more
than 4 students at a time even though the equipment was there. (June 2007,
Japanese)
It is essential to note that smooth running of the exams will depend on technical
expertise at the school and/or district levels and students will be at the
mercy of this support system.
All this creates extra and unnecessary stress, work and hassle for questionable
outcomes.
• Increased load without compensation for grade 12 teachers
As mentioned above, the grade 12 teachers have to help with the administration
of these exams. It is completely unrealistic to maintain that they can leave
this up to “invigilators”. Indeed, who will train and pay for knowledgeable
invigilators? Japanese teachers (even the technically “savvy” ones)
found it a very onerous and arduous administrative process. In addition, teachers
then have to mark the exam speaking components on their own time.
• Inadequate Ministry technical support
According to reports from teachers, the tech support actually given by the
Ministry during the Japanese and French Immersion e-exams left much to be desired.
It
did not go as smoothly as the Ministry is reporting. (The technical support
entity for the Ministry answered about 700 calls for Japanese alone – an
exam taken by 700 students.) The majority of Japanese teachers reported some
technical
problems. Among the complaints were the following:
- students had to redo their speeches due to a number of technical problems,
- the lines were either busy or no one answered when calls were made re technical problems.
We can imagine what will happen when all the French students come on board
in addition to Japanese and Spanish!
• Other points:
1. Many question the notion of having a “provincial” exam that the
students’ own teachers actually mark (i.e., the standardization and the
objectivity will be lacking).
2. The exam has a window of 3 weeks before the rest of the provincial exams.
This takes away valuable time for teaching and practice, especially in a semester
system.
3. Have the cheating possibilities been adequately addressed? Our experiences
as teachers show us that high stakes exam situations entice students to resort
to any means possible to achieve success, even if dishonest. Students are highly
sophisticated when it comes to cheating using technology.
Implementation Process
The implementation of oral exams is a major concern of teachers from all languages
and has been expressed numerous times in letters by the Japanese teachers as
well.
• Lack of opportunities for dialogue or input from experts in the field
This is a momentous issue and yet no public discussions took place. No consultation
with informed leaders in second language education in the province was conducted.
(The Ministry had received many letters in the past requesting an oral component
from professional groups of coordinators and teachers. None of these groups
was consulted.) We are not sure whether exam committees were consulted as to
the
format or the type of tasks. If they were, it is definitely not sufficient
for they do not represent a broad enough band of expert teachers in second language
education. Such momentous decisions need a team of highly knowledgeable master
teachers, language education leaders, and assessment experts whose business
it
is to work with curriculum and assessment.
In a very real sense teachers (and, as a result, students) were blind-sided
by this addition to the exams. No one was aware of any piloting or field-testing.
The field was suddenly informed of this. For the most part this “informing” consisted
of putting the information of the new component on on-line exam specifications.
Not only is it hard to see the full extent or ramifications of the changes in
that format, but giving informed feedback is very difficult, if not impossible.
Having enlightened, wide-spread discussion isn’t possible either. As
a result, teachers in the field who are dealing with the students on a daily
basis
feel that the process has been imposed upon them without proper discussion,
consultation, input or preparation. Indeed, any input or legitimate queries
seem to be disregarded.
The teachers desire an efficient and methodologically/communicatively sound
way of assessing oral competence.
Even after the Japanese exam last June, no discussion is being entertained
re the effectiveness of the process. In fact, when concerns are expressed they
are
not acknowledged or are even flatly denied in certain meetings or communications
with the Ministry. This is not a good situation for our students or our teachers.
They deserve better.
• Lack of pre-exam general piloting (as far we know)
When an exam component appears on the web, there is an assumption that the necessary
research and piloting have already taken place. Such formally mandated directions
generally indicate that all the issues have been thoroughly examined and adequately
reviewed. We find however, that this is not the case here, for example, anchor
samples are still not available and piloting data of the rubrics is not public.
Our concern is that the mandate may be premature and the process flawed. As
a result, we fear the very notion of oral exams may be abandonned. We wish to
emphasize
that our preference is not to abandon oral exams, but rather, to see a more
pedagogically sound and more practical method be adopted.
•
Limited introduction to the process – Japanese
We are not sure what the intention was in starting the process with the Japanese
students. If it was to “pilot” the process, then there should be
extensive and public reviews of it with open discussion and modification as
necessary. This will affect all language groups and all should be involved
in the discussion.
Technology
• Schools need to acquire new equipment.
This is a money and a time issue. Who will pay and who will do this extra work?
Will the new equipment have use beyond this exam and therefore be worth the
purchase? What real educational value does the use of this technology have beyond
meeting
Ministry e-exam requirements?
• Technology adds a further level of complexity and stress to the examination
process.
Students and teachers have to jump through technology hoops to assess simple
oral tasks. It is stressful for teachers and students because they are worried
something will go wrong that they cannot control. And things do go wrong as
evidenced in the Japanese exams. When it comes to technology, glitches are inevitable.
This became amply clear when Ministry representatives tried to present the
websites
for these exams in a teacher workshop and the equipment did not work.
• Technology is a tool that should support best practice, but not determine
it.
In this case it is not supporting best practice, but rather, limiting it. We
have better, more authentic and more efficient methods available. Again, informed
and open discussion by all concerned (Ministry needs of tracking, teacher needs
of marking, student needs of fair assessment, curricular needs of authentic
communication, etc.) could bring to light a tenable and fair solution for all
the facets that
need to be considered.
• Disadvantaged students have greater difficulty accessing practice
opportunities.
Students who do not have the appropriate equipment at home for practice are disadvantaged.
Practising at school is not a realistic solution to this very important issue.
For one thing, schools are not always equipped or able to provide this practice
time.
Conclusion
Although technology is an issue, it is not at the heart of the concerns for
teachers. If the technology route proposed were indeed the best method for
assessing the
oral component, any difficulties there could be overcome and tolerated. However,
the pedagogical concerns are compelling, especially in light of the fact that
we already have a superior method of summatively assessing oral competence
that has been promulgated by the Ministry itself -“Assessing Speaking
Skills in Core French”. (Grs.
8 to 10 and Grs.
11 and 12). This process was reformulated for the other languages as well,
but when personnel changed at the Ministry level and the oral exam became a
more and more distant
reality, the work done was also put on the shelf. Perhaps the creators of the
concept of this e-exam component were not sufficiently aware of the extent
of the use and efficacy of the process that had already been created for this
very
purpose.
We favor the “Assessing Speaking Skills in Core French” procedure
as a summative provincial assessment tool for many reasons. (See Appendix I)
The only equipment necessary is a digital video camera to tape the conversations/discussions.
The DVD could be sent to the Ministry for perusal/marking. If teachers have
to do their own marking anyway, then this is definitely a better way, for it
involves
much less time and set-up. Marking can also be done effectively on small inexpensive
computers. Students are seen and the important non-verbal aspects in communication
are visible. Standardization, objectivity (by having other teachers mark) and
tracking are all possible. It is already being done in large-scale language
exams. This seems to us a superior way of using technology on all levels.
It is paramount that more informed consultation, discussion and problem-solving
occur before implementing something so important. This would provide opportunity
to address the real issue of pedagogy and what is best for our students. Sincerely,
APPENDIX I
The “Assessing Speaking Skills in Core French” process was created
for summative evalua-tion of oral skills to reflect the communicative IRP Learning
Outcomes. It underwent wide-spread piloting all across the province and had
assessment experts involved in creating the rubrics (from grade 8-12). Teachers
bought in
to it because it was an open and accountable process that was not immediately
connected with high-stakes testing. Forums for feedback were conducted and
input was acknowledged and implemented. Based on that open reaction and response,
the
process underwent many modifications and refinements. Many teachers tried it
while doing action research courses. Not only did it change practice because
teachers could experiment with it and experience the efficacy, but a new excitement
and positive energy came into communicative teaching. Unfortunately a change
in Ministry personnel and lack of response on the exam front shelved the further
work to be done for all the languages.
The method
• Encourages spontaneous oral competence
• Inspires oral confidence in the students
• Generates a high degree of real success
• Is appropriate for any level and theme
• Scaffolds oral teaching well
• Simulates real-life situations
• Encourages higher-level thinking skills according to Bloom’s Taxonomy
• Has leveled rubrics that are easily internalized
• Is easily implemented (highly supportive of IRP communication LOs)
• Is efficient – can be done in classroom time, marking on the spot
• Reflects and supports classroom reality
• Requires no equipment (except in a summative provincial exam situation)
It has been argued that AP exams use the individual student model. It does
not mean that that is the best model, and the curriculum for AP is not the
same as
the BC IRP. In fact, when people hear about BC’s “Assessing Speaking
Skills in Core French” model, they are impressed. Publishers have used
it to design better oral tasks for their programs. It should be noted that
the total numbers in AP are very much smaller than in our regular language
classes.
In response to the answer given in the FAQ
#14, i.e. why a presentational format
rather than an interactive procedure was implemented,
we would like to point out that the individual learning outcomes need to be
interpreted in the context of communication in general: “Students are
expected to engage in increasingly complex and spontaneous oral interactions
in which they demonstrate
their facility with Spanish and the strategies they have developed to sustain
and extend communication.” This is the contextualizing statement for
the Spanish 12 Communicating Learning Outcomes (pg. A-55 IRP -2005). Sustaining
and
extending communication means, in part, interaction. The previous points we
have made in terms of pedagogy still stand.
We also strongly disagree that “a presentational format is easier for invigilation
and can be better controlled in a standardized setting”. We have experienced
great success with the interactive model in the classroom and in standardized
formal testing situations.