E-exam Survey

In late November 2007, teachers were invited to complete an on-line survey hosted on the BCATML website. This was part of an ongoing strategy to respond to and quantify/articulate the concerns expressed by language-teaching colleagues throughout the province. There were 114 respondents between November 24th and December 12th.

According to the survey's findings, the top three concerns related to the e-exam are:

1. Amount of input and piloting prior to implementation 79.5%
2. IT demands on teachers, schools & districts for exam administration 77.2%
3. Increased load (without compensation) for Grade 12 teachers 77.0%

53 of the survey respondents also wrote comments about the e-exams both from the perspective of having administered them (Japanese 12) or preparing to (Spanish 12, Mandarin 12, French 12). They are available below.

Two meetings with Ministry Exams Branch officials (one including a BCTF representative) are scheduled in January and one in February.

E-exam Survey Comments:

1.The most frustrating part is that the exam does not match what we have been doing in the classroom since grade 8: promoting spontaneous oral conversation about authentic themes in a partner set up. It invalidates everything we have been working towards and have been 'mandated' by our adherence to the Ministry's own IRP. When you look at the French Challenge exam, one wonders why this exam differs so greatly. There is already a tried and tested (for over 10 years) method of examining orally that also has a tried and true evaluation method which does not require the degree of technical difficulty as the proposed e-exams. It makes the teaching of our Core French a much less authentic/engaging practice. You are going to turn teachers off just when we need them to choose Core French! Sat, 11/24/07

2.I think it's going to be an important part of the exam process. Including an oral component shows that we value communication as well as written skills. However, these tests do not seem ready, and schools are not ready to deal with this task. The trials that I have seen have had numerous glitches (all of the answers being A, systems freezing, etc). Why force it when it's not ready yet? We should take our time, make sure all schools are technically ready, and design questions that aren't just going to reward the top working students that can write a paragraph and memorize it. We should have marking teams set up so that the marking doesn't fall on the teacher - not just because of increased work load, but because of subjectivity, which is supposed to be the whole point of a provincial exam. If it's just marked by the teacher anyways, why not just rely on our school marks as a valid mark of oral work learned? Sun, 11/25/07

3.Why aren't field experts consulted in matters with such significant effects on students and teachers? Sun, 11/25/07

4.I am very disturbed that such a enormous undertaking was done seemingly without adequate input from the people who would know how to most appropriately test students...the teachers. Had proper input been sought from us, there possibly wouldn't have been as many administrative/technical problems. I truly value and oral and aural component on the Provincial Exam; however, if it is not designed in the most effective way, I would rather not include these components. We must rethink the current exam design because it is definitely flawed on so many levels (see the numerous letters that have been sent to the Ministry from concerned teachers from all languages). Sun, 11/25/07

5.I want to be able to test all my students at the same time ! Mon, 11/19/07

6.Why hasn't the oral pilot exam process been integrated into the e-exam? It is a process working well in many second language classrooms. It's working for the Challenge Exam process AND it matches the interactive, communicative focus of our curriculum. Where was the input from the field? Sun, 11/25/07

7.Marking should be centralized like reading/writing components to maintain fairness and marking standards. The class size affects computer availability and teachers' exam administration. Naturally, if we have a big class, we end up with too much stress and little preparation time for the students. The speech can be modified. For example, students can impromptu describe the picture or making up a story, instead of talking about one of the topics prepared beforehand. Sun, 11/25/07

8.L'examen du fameux Bac en France,est administre a l'oral par des professeurs competents en la matiere et non par des machines!Que le gouvernement depense donc un peu d'argent, et qu'il envoie des profs de francais dans chaque District! [Why doesn’t the gov’t spend a little money to send French teachers into each district to assist with exam administration?] Sun, 11/25/07

9.Negative Points:
a The Ministry almost completely disregarded the concerns of Japanese teachers
b The practice exam sessions had to be held outside of regular class time (uncompensated extra work for teachers)
c I had to phone the Ministry 20+ times during the actual administration of the exam due to technical problems (a cell phone is mandatory)
d The monologue should probably be dropped but wasn't as bad as expected due to a reasonable marking rubric
e. The administration at my school thought that the administration of the exam was solely the responsibility of the Japanese teacher (I had to do 100% of the administrative duties including practice sessions outside of class time)
f Many students had trouble doing the practice test at home (& only had access to 1 set of questions)
Positive Points:
g The marking was very easy and doesn't pose much of a burden given the schedule at the end of the year
h The questions on the Japanese exam were generally fair and reasonable (a concern is that the Ministry might ramp up the level of difficulty over time Sun, 11/25/07

10. What if there are technical difficulties during the exam? Students overhearing the responses of others during the oral component? Noise level "cutting and pasting answers" training and practice opportunities during the year. Are there practice tests we could do on-line well before the exam date? Sun, 11/25/07

11.Lack of discussion with Ministry and disregard for teacher input Sun, 11/25/07

12.The session at the conference for Spanish teachers about the new exam with the rep. from the government was not adequate. He did not give out any handouts with rubrics, or website information for us to use. We could not see his overheads from the back of the room. I think we needed a whole day for the discussion of the exam rather than an hour and a half. I understand that he was just the "messenger," but why can't the government be more prepared- they're dealing with teachers! We NEED to be prepared and trained appropriately in order to give the students the best chance possible. I really feel for teachers in schools that don't have enough access to computers in order to teach the students and for the students to practice. If the government wants to implement a new format of the exam, they should be warning administrations of schools and teachers WELL in advance and supply all the necessary criteria and training! If teachers have to mark the oral part, will we be told we're marking "wrong" by the government?. Sun, 11/25/07

13.As a Japanese teacher I am convinced that the only constituency that the Ministry will listen to is parents and students. If it is proven that the exam is detrimental to student grades and will reflect negatively on the province in any way they will be taken off. Otherwise they do NOT care - this government's style is to make policy and download all costs and responsibilities. Sun, 11/25/07

14.Teachers of these courses should have some input in the implementation of these e-exams. All concerns should be addressed before actual implementation. Discussion and thoughtfulness should precede implementation, don't you think? Sun, 11/25/07

15.I am a Spanish teacher with no grade 12 class this year. Do I qualify for training this year or will it be available again when I have a grade 12 class? The classroom management of prepping students for an online exam while teaching in a regular classroom. Who supervises the students as they practise their oral component (I don't have computers available in my classroom) ? How will they manage the exam during regularly scheduled class time if there isn't any lab time for them to take the exam online? Do we stop computer classes three weeks early so that language students can have access to computer labs? Why bother with provincial exams if the universities don't use them for entrance/placement? We could put the money saved from abolishing the exams into the classroom. Sun, 11/25/07

16.I am very concerned about the risk of cheating for this exam. There are so many ways to cheat on this exam it is a miracle if a student DOES NOT cheat! I am also concerned about the availability of training for this type of exam. I went to a training session last year, even though I was on maternity leave because the issue worried me so much. Because I did not run the exam last year, I still feel that I will be new to the process and therefore feel just as anxious about it. If we mark our own students, how do we know if we are giving them a fair deal? If we mark easy all year and easy on the exam (or the reverse) then how will we know if we are "on target" across the province? Isn't that the whole point of province-wide exams? Sun, 11/25/07

17.Some of these issues can be handled with extreme planning, hardware purchasing and extra teacher help and release time. But, after all those resources are made available, it can all still fail if something happens to our internet connection - which happens OFTEN, for no discernable reason, and nobody can predict WHEN it will happen. Sun, 11/25/07

18.My biggest problems with administering the Japanese exam last year were: 1) Monologue: it is not a real assessment of communicative ability. It ended up being a memorization task. Students ended up putting a great deal of time and effort on a task which doesn't help them to improve their communicative ability, doesn't effectively assess their communicative ability, and took time from other studies. 2) Computers freezing during the exam. It simply made it difficult to administer, and some students had to repeat spoken answers, which takes away from the spontaneous nature. 3) Marking: Each teacher marking his/her own students seems to be counter-intuitive to a provincial exam, which is meant to be impartial. If the Ministry cannot provide markers as in other exams, they should at least randomize the samples and re-distribute them amongst teachers so no one is marking their own students. Mon, 11/26/07

19.The technology and availability of computers is a major concern. Mon, 11/26/07

20.I am a Japanese 12 teacher and administered the e-exam last June. Aside from the technical problems (background static on recordings; students 5 computers away still overheard each other; spent $300.00 of resource budget for headsets; class time spent/lost preparing for exam; shall I continue?), the most pressing concern I have is the unnatural 1-3 minute monologue with a computer. I don't teach that skill to my students. I think it would be difficult in one's native language, especially as you are being evaluated on your responses. Lastly, they ask me to mark it! I've been doing that for years as a one-on-one oral evaluation with much greater ease, and much less anxiety all around. I wish the "experts" at the Ministry would actually listen to the real educators on the front lines and make some appropriate decisions based on their suggestions, and not the other way around! Mon, 11/26/07

21.In addition to the above, I am concerned with having students speak to a computer. We do not do this in the classroom, so why should they be doing this on the final exam? Mon, 11/26/07

22.Monologue is not communication. Answering a bunch of unrelated questions is not communicative. A test has to test what we teach in our classroom, which is communication in our modern language program. Mon, 11/26/07

23.I understand the attempt to provide the aural and oral components. However, the technology is not in every school. We have not been given the resources by the Ministry or ample time to ensure that the technology is in place in time. In time means having students access it once the course has started and time for teachers to practice using it before the course has started. Even the Ministry's implementation seems last minute. Furthur we do not know if our classroom listening resources are sufficient for the provinical. For the speaking component, why are there not recorded samples so we can prepare our students to the appropriate standard accepted. We all want to do the best for our students. I feel ill prepared with the resources provided by the Ministry. Mon, 11/26/07

24.What if the students do not know how to type properly. 'Typing' is no longer a course and many students type with only two fingers. It will take them forever to type out the French essays. Mon, 11/26/07

25.I thought last year's Japanese 12 e-exam did not assess students' abilities to communicate in an authentic manner. This is not how we assess our students throughout the year. It was very stressful for everyone involved. Mon, 11/26/07

26.If I have a concern it is with the amount of extra time that Language teachers must put into the set-up and the administration of the exam itself without compensation. For example, in my school the admin is not interested in helping out with the administration of the e-exam AT ALL. They have made it very clear that they think that it is a teacher responsibility and that it is our job to coordinate with the tech support. Despite this, we are still expected to take our turn invigilating the other exams too. Where is the BCTF in this? They seem to have other priorities. For what exactly am I paying dues when all of this is going on basically under their noses? Mon, 11/26/07

27.I am most concerned about three issues: 1) level of technical support ( I am not a technie! and have great fears about exam day glitches) and access to the technology required to deliver an e-exam 2) level of training of teachers in the marking procedures and exam procedures prior to being obliged to use the e-exam format 3)(lack of) availability of practice exams for the speaking component well in advance of students actually taking the e-exam Mon, 11/26/07

28.Students had enough speech and conversation experiences in class. They don't need to test them. Teachers became busier and busier because of this oral exam. The exam results are not showing the true ability. The exam was not checking the level of student's ability. Most important element for learning language is not checking through the computer. It is very unnatural way. Man to man is better. Teacher and students should talk in class as we used to do it. The exam is just giving the students a fear. Anyway, my students did well, but I still that it is not a good measuring for students’ ability. They had enough time to be judged their speeking ability in class. We are teaching language therefore students have enough tests for the oral part of course during the year. One more thing is that teacher should not be told to mark the exam. It is very hard to finish everything in the busiest time. Mon, 11/26/07

29.This whole notion of a legitimate evaluation and legitimate content is absolutely ridiculous. It is in my opinion, a "make-work project" for teachers and an additional and unneeded stress on students. To date, valid and useful Ministry support has been virtually non-existent and, based upon recent indicators, that is not likely to change. This whole operation can be summed up in two words: not working!!! Mon, 11/26/07

30.When computers freeze, students have to redo that portion of the exam, so the time limits are meaningless. Computers froze on one-third of the students at one or more sections of the e-exam in June. If teachers mark their own students, then how is the Provincial Mark different from the School Mark? We might as well increase the weight of the School Mark to 70% (instead of 60%) and omit the oral components in the Provincial. Tue, 11/27/07

31.I am wondering why we are investing so much energy into these exams when our local universities are talking about not incorporating the provincial exam marks in the application process. Tue, 11/27/07

32.Monologue is very difficult to do even with your native tongue. Students do not practise monologue as oral and aural exercises. Tue, 11/27/07

33.The SATML submitted a letter recently outlining our concerns in a very articulate manner. Please refer to it. Tue, 11/27/07

34.I welcome the testing of communicative skills in the exam process, but feel that the format relies too heavily on technology, and does not plan for the upgrading needed by most schools. There was also very little communication about the process beforehand, giving teachers little time to prepare for the exam (I feel especially sorry for the Japanese teachers who had only months to prepare). While the use of technology for this exam is not completely out of place, a much "lower-tech" form would have been possible and better suited to the level of technology available in BC schools. One huge technological issue is a conflict with the province's access permissions: on one hand, the students need to download ActiveX controls for the exam, but on the other hand, the province's computer protocol does not allow students the security level to download ActiveX!! I am not concerned about cheating, because the exam is still testing things like pronunciation and fluency, which can't be "copied" from another student. Tue, 11/27/07

35.Main concern is the lack of connection between communicative pedagogy and this style of assessment. At the same time, the administration and tech issues are a nightmare! Wed, 11/28/07

36.These exams are of great concerns. The exam is not suitable for our school environment as it struggles with technology in general and I doubt that many schools are technologically advanced either. Why is the Ministry doing this to teachers? I would appreciate the exam IF it was remotely possible for me to administer but it is NOT! Wed, 11/28/07

37.We need technical support to administer the exam. Students and teachers need time to practise this kind of format. Would the government provide this kind of service? Wed, 11/28/07

38.Marking of these exams is a concern, in at least these two scenarios: 1) at schools where there are no teachers of the target language -- what is the mechanism to farm the marking out to other markers? Are teachers at nearby schools appointed the task? By whom? For what recompense? How are teachers chosen? Is this system fair or equitable? Is this system open and transparent? 2) at schools where there are one or two teachers of the target language. Are teachers "told" to mark them? Are they asked to "volunteer"? What measures are in place to ensure that undue pressure is not applied by admin to "encourage" volunteers? What if all teachers at a given school, or in a given district decline to volunteer? What measures are in place in the event of a computer crash, or an internet service interruption? Wed, 11/28/07

39.While I applaud the idea of adding a speaking component to the exam, this format is poorly thought out and does not involve students in real speaking situations. A monologue is not an authentic speaking situation and does not reflect the goals of the IRP. I strongly object to marking my own students without training and/or compensation. This whole plan is poorly thought out and concerns are being glossed over by the ministry. This is not just about technology, this is about poor pedagogical practices. Wed, 11/28/07

40.I believe it is the responsibility of the ministry to ensure that each secondary school is equipped with enough computers for e-exams. There might be as many as 3000 students who will take the Man 12 provincial exam. 1500 of them do not take Mandarin at school; Man.12 challenge marks become their school marks. Will there be training sessions/workshops on e-exams for Mandarin teachers in the public schools system? Will the same training sessions/workshops be open to all the hundreds of "Mandarin teachers (regardless of their teachers' qualifications)" who are tutoring students to prepare for the Mandarin Challenge Exams in the heritage Chinese schools? Who will be marking the e-exams of all Mandarin 12 students? Can the Ministry present the document on e-exams to all language teachers in the province? Wed, 11/28/07

41.The lack of consultation PRIOR to implementing the e-exam and the COMPLETE disregard of the feedback given to the Ministry by the Japanese teachers. Apparently the Ministry people told the Spanish teachers that all went smoothly with the Japanese teachers and their e-exam. That was NOT the case! Wed, 11/28/07

42.Not only am I the only Spanish teacher at my school, I am the only one in my district. We also have no language coordinator. I do not feel like I should be responsible for technical details of preparing for the exam and yet I will need to be greatly involved. I also wish that I had a colleague with whom to share ideas and support in preparation for this exam. Many schools in larger centers have native speakers who take these exams and classes in order to have an 'easy and successful' course. Fluency of these speakers will be at a completely different level than for students taking the language as a second language. I do agree with the concept of testing students' oral competency - but strongly disagree with a monologue. Students do not even have to do that in English! I also feel rushed and unprepared with the format the exam will have. I would like to have lots of training before I have to begin teaching the course. Initial reaction from students is: "This is going to be so hard, I think that I am going to drop this course." That comment concerns me greatly. I do not feel that we should teach to the test - but rather be teaching for success. If students won't take the class because of the exam, then we are not very successful in what we set out to do. Thu, 11/29/07

43.Not every school is equipped with enough computer labs. If the school has a large Language 12 program, there won't be enough space for all students to take the exam at the same time. How do you solve this problem? Thu, 11/29/07

44.I believe that it is grossly unfair to assign teachers to mark this exam without financial compensation. Thanks Thu, 11/29/07

45.Obviously all of the issues on the survey are of great concern. Regarding exam format, the student monologue presents the biggest problem. First, it is not communicative by nature. It does not reflect classroom practice and it is a simple task of memorization...not a test of ability to communicate in another language. Furthermore, the fact that the classroom teacher must evaluate this task is a big problem. First of all, standardization and teacher bias are completely neglected here. Regardless of the fact that there is a standard rubric of evaluation, the classroom teacher must not do this evaluation. Some teachers in the province teach 1 grade 12 language course whereas others teach 3 or 4 grade 12 language courses. The extra workload incurred from the marking of the oral component will be greater for some and less for others...this is unfair...especially where compensation is not being considered. One other area of concern is the training and in-service for these exams. The classroom teachers need to be trained on how to use the new technology. Furthermore, if they are to mark the oral exams, they need to be trained on the marking practices...just like all the other provincial exams where the teachers are first trained. You can't hand a rubric to the entire province and expect to get the same result. The interpretation of the rubric from teacher to teacher can vary...each teacher has a vested interest to have their students do the best that they can. This is why the exams should NOT be marked by the classroom teachers. The exams need to be marked by a properly trained committee in order to obtain consistency and standardization. This is no different than any of the other provincial exams. This is the point to standardization. If it is expected that classroom teachers will mark this exam, then a compulsory training session (paid, on school time etc) must take place for every single language 12 teacher in the province weeks before the exam. Thu, 11/29/07

46.The questions of fairness, smoothness of operation and amount of extra work for teachers involved are what concern me most. Fri, 11/30/07

47.What are the pedagogical reasons for having e-exams? There doesn't appear to be any advantages of e-exams over the existing Core French Oral Assessment process, so why fix something that's not broken? I have a huge concern that e-exams are "imposed" simply for the sack of appearing "technologically enhanced" or "modernized based on computer technology" but has NO language acquisition or communication benefit. Sun, 12/2/07

48.One of my biggest concerns is teachers marking their own students. Once this occurs, the exam ceases to be a legitimate exam in terms of a provincial exam. If I have to evaluate my own students, why would I need jump through all the technology hoops and spend the extra time required to prepare students for this type of exam, just to do what I'm doing anyway? It is absolutely absurd. Sun, 12/2/07

49.More added burden for the grade 12 classroom teacher to mark these exams without compensation in addition to other exam invigilations during exam period.teachers are already too stressed with too many responsibilities without having to do more !!! monologue...set up is so different from classroom oral testing as well as the language challenge exam where it is in pairs or trios....not monologues. the individual provincial testing does not reflect what we do and test in the classroom. what is meant by "disadvantaged students?"...special needs students usually are exempt from taking a second language Sun, 12/2/07

50.This exam form is not suitable for the following reasons: - the speaking component does not meet the IRP expectations (not a 2 way conversation with computer) - the logistics of piloting, proper training and administrating this exam are not realistic - even if all schools in the province were equipped with proper computers, this exam format has too many flaws (see items above) and requires too much extra time, money and effort Wed, 12/5/07

51.I am most concerned about the lack of preparation for teachers. We get no information of support unless we do it ourselves and it's easy to miss newsletter and such. Also, our schools' computer labs are notorious for malfunctionning. How does this play out on exam day - a major crisis for the teacher? Thu, 12/6/07

52.I am a Japanese teacher who has taught at a public high school in B.C. for 17 years and I am concerned about the e-exam. Last year I experienced firsthand the e-exam with my 11 students (8 + 1 walk-in student took the exam). I consider myself a dedicated, knowledgeable teacher of Japanese and I found it a lot of extra work and a lot of pressure trying to prepare my students for the e-exam component. I am extremely worried about teaching Ja12 next semester as I have about 26 students enrolled (3 times the number last year). Only with 9 students last year, I had to retest 3 students due to technical problems beyond our school's control. My former vice-principal, a computer expert now at the Board Office this year, could not even prevent this. In short, I have deep concerns about this e-exam in all the areas you mention above. Major revision needs to be done (e,g,: have mandatory oral components like a speech or an interview in the school mark (This way, it's much more realistic as students are presenting to a real, live audience instead of a computer) in lieu of the one day monologue portion). The listening portion could easily be done in a classroom with students filling out a scantron as they listen to a tape/CD instead of in the computer lab which is extremely difficult to book since we have about 1400 students. This may also avoid all the computer glitches, as well. In this way, the e-exam does not have to be conducted using computers and can still be included in the provincial exam mark. Furthermore, without sufficient support, time, and resources allocated to this, it adds unnecessary stress to everyone not only for teachers, but administration and students. I could continue, but I still need to prep and mark. I needed to fill out this survey, as I feel there is a dire need to make revisions to this e-exam. I hope I have given some constructive opinions and you will listen to my "small" voice. Mon, 12/10/07

53.I think we are going to open Pandora's Box by allowing Modern Language Teachers to mark grade 12 exams. Exam markers apply to mark and are trained and paid accordingly. What's to prevent the ministry from forcing us to mark all of the exam? Moreover, if I'm to mark my own students' exams, I want them to do well: What is to prevent my giving them all A's? As well, the long window for students to gain knowledge of the oral exam questions allows for cheating. From what I've heard, the BCATML must take a stronger stand against these exams. Why continue with a flawed process? If we stand together, along with the BCTF (is our union even aware of this situation?) and other teachers' associations, we can delay this proposed method of examining. A few years ago, there was the prospect of Social Studies 11 teachers marking their own students' essays in the provincial exams. Well, the upswell of opposition on the listserves and a strong stance by the BCSSTA ensured that the exams would be marked by those who put their names forward as to be markers. I urge the BCATML to do the same. Mon, 12/10/07