Forum Developments in ongoing English classes using electronic courseware for multi-level repeaters

MomoyamaGakuin University mgrogan@andrew.ac.jp In compulsory university English classes, not all students pass. For a variety of reasons, a number of students find themselves needing to retake (repeat) one or more English credits. Placing such students in a class can be difficult for administrators. In 2008, a pilot study was launched using DynEd, a commercial software suite designed to support students’ individual learning needs. The software allowed for a large number of students of different ability to study a wide range of materials at levels matching their individual ability. By studying on a computer, rather than face-to-face, they could also study in the same time slot (class). This paper reports on how class implementation has developed and matured, particularly since it was first presented at the JALTCALL 2010 Conference.

In compulsory university English classes, not all students pass. For a variety of reasons, a number of students find themselves needing to retake (repeat) one or more English credits. Placing such students in a class can be difficult for administrators. In 2008, a pilot study was launched using DynEd, a commercial software suite designed to support students' individual learning needs. The software allowed for a large number of students of different ability to study a wide range of materials at levels matching their individual ability. By studying on a computer, rather than face-to-face, they could also study in the same time slot (class). This paper reports on how class implementation has developed and matured, particularly since it was first presented at the JALTCALL 2010 Conference.

The context: a review
DynEd is a software suite (functioning much like a collection of textbooks) for schools and universities. The software is being used for courses at a medium-sized university, and is administered through a Language Centre. The Centre administers all compulsory English classes at the university. DynEd was chosen for repeaters' classes for a variety of reasons (see Grogan, Silsbee, & Bell, 2011). To review, the main goals of these DynEd [repeaters'] classes were to reduce the number of classes needed for repeaters in the timetable (allowing teachers back to mainstream classes) while simultaneously trying to provide course material that would both broaden and increase language ability. Unlike in previous classes, students could finally be given concrete course goals and objectives, and the means to study them flexibly, at a level to suit their individual ability.
In the fall semester of 2008, two fulltime contract teachers ran a trial test of the software with two classes. This trial was mostly aimed at assessing the technical needs involved in such an undertaking. In 2009, a subsequent full-year trial focused on assessing classroom and student needs. In 2010, at the writing of the last article, the DynEd classes were about to officially begin. Four part-time teachers and four full-time contract teachers taught these classes. Since then, a number of changes have occurred at the university level, which have had varied effects on the administration of the classes.
When the repeaters' classes started using DynEd, it was the only choice for students who had failed their regular English classes. Up until this time, school policy was that repeating students were not allowed to enroll in regular classes to meet their English requirement. Although there is no empirical evidence in this case, it has been shown that Computer Assisted Language Learning does not appeal to everyone (Stepp-Greany, 2002). It was thus decided that students who had failed an English class would be allowed the choice of taking a class using DynEd or enrolling in a regular taught class. It should be noted that in the first semester of 2011--the first time students had had such a choice--more than 60% of students chose to study using the software. There is no data available as to why students chose to enroll in the DynEd classes or re-take mainstream classes, but discussions are currently underway as to how such information could be gathered. However, exit survey data is regularly taken from DynEd classes, and remains consistent in supporting the efficacy of the software among the students who complete the classes.

Teachers
In the first full year (2010), classes were taught by four full-time contract teachers in the Language Centre, with one teacher performing the duties of lead administrator (e.g. creating class lists, supplying handouts, general maintenance and evaluation). Over the next two years, an extra contract teacher was brought in and two were lost, bringing the number to three. Administrative duties have since been divided among two of the three teachers in order to share the workload and provide some redundancy in the systems.
Although regular classes have a curriculum, implementation is largely left to the teachers in charge. This is not the case in the repeaters' classes, with a large number of students taking several classes simultaneously. The structural organization and process of these classes have to be consistent, even though content may vary. In the pilot, only full-time teachers were involved, making this relatively simple. When DynEd classes officially started, the addition of part-time teachers gave rise to a number of issues that have been addressed in earlier literature (e.g. Bingham & Larson, 2006;Kabilan, 2009), the most prominent being developing teachers' understanding as to how the software operates and developing a system of student monitoring and evaluation that was easy for teachers to use and understand. In order to make sure all classes were uniform in implementation, training was given to part-time teachers before courses began, and meetings were held whenpossible before semesters begin. Due to the large amount of material teachers need to be familiar with, both in terms of the software and administratively, regular emails were exchanged, and many part-time teachers had support during the first few classes.

The software
DynEd offers a series of courses, with each course functioning like a textbook. Students navigate their way through a course by means of repetition and comprehension checks, with tests scheduled after completion of material. Using the software features, students are expected to actively record and check their spoken output with the language they hear presented in the course in order to aid acquisition. Content of any particular course is computer-adaptive, so choices will be made easier if students appear to be having difficulty, and more varied language will be introduced when students appear to be doing well.
Courses vary in their focus and content, with each course divided into units or modules of study. The software measures how students study to produce a "Study Score". This is calculated using variables such as frequency of study, use of function buttons (e.g. repeat, record, translate, etc.), progression through the course, and test scores. Positive numbers represent good study practices, while negative numbers represent poor study practices. In order to facilitate better study habits, advice is given to students using the "Intelligent Tutor". This gives students specific advice based on their study records as to what they are doing right or wrong in terms of studying the material, and how they might improve their score.

Course requirements
The overall system is concerned with improving individual student's language ability. For this reason, each student only gets one log on ID. Pedagogically, this means that we can monitor breadth and depth of development. It also means, however, that administrative care beyond what the software is capable of is necessary to marry the course requirements to the university credit system.
The current requirements for each class stipulate that students study for a minimum of fourteen hours, pass the Mastery Tests that are required for the course being studied, and finish with a study score of +2 or more. Additionally, students are generally required to achieve 80% completion for any given section to ensure that all test material has been sufficiently practiced.

a typical semester
Following the many changes that have passed since officially starting, a pattern has been established. The following describes the current course of a semester. Although some details may change in the future, this is roughly how the system works at present.

Set up
For the Language Centre teachers, most of the work comes a week or two before classes start and into the first two weeks of classes. Before classes, orientation materials must be modified to reflect any changes in the software or in the university management systems. The teachers then work with both the staff at the Language Centre and the General Affairs department (which is responsible for student registration) to set up class lists for each teacher and import them into the DynEd system. Because each student receives a single ID regardless of how many classes they are taking, a master student list for all classes must be made and then broken into separate lists for students taking one DynEd class (single class students) and those taking multiple DynEd classes (multiple class students).
Students who have done DynEd before are marked on this list, as these students already have an ID assigned to them. Reviewing the student data helps to decide if they are familiar enough with the system to study independently or need to retake an orientation session. One merit of the DynEd courseware is that there are several courses offered at similar levels, so finding an appropriate course for such students is fairly manageable. Students who understand the processes involved can begin studying almost immediately, while teachers spend more time with new students.

Lesson 1: Main orientation
Because registration is open until well after class begins, the class lists need careful monitoring and updating. It is common to have students attending class who are not yet shown on the class list, so taking attendance is a good first step for the first few classes. Students without an ID cause no issues in the first class, but they cannot begin the study process until they have an ID. Follow-up on these students is essential.
During the first class, the students receive orientation material outlining what is required of them in general terms to pass the course (as outlined above), the expectations teachers have of students, and an overview of how to use the software. To facilitate this last part, students are given dummy IDs for the first class. Using these IDs, students are shown a low-level unit. The purpose of using a low-level unit is to allow students to focus on the technology rather than the English skills involved in the course. Students are introduced to the lesson navigation features, and how they use these features to learn. Study practices, such as simultaneous shadowing, are also introduced. Finally, students are shown the "Speech Recognition" function on DynEd, and can see how the computer recognizes (or doesn't recognize!) their speech.
The Study Score and Intelligent Tutor systems are also introduced. As students practice, they can see their study scores go up or down, depending on the quality of their performance. The Intelligent Tutor provides advice on how to make the scores better. This handson experience shows how the system works and is part of the groundwork for the initial stage of the course.

Lesson 2: Rules and placement
Lesson 2 begins with a true/false-style quiz on the orientation material covered in lesson 1. This gives students a chance to clear up misunderstandings, and those students who missed lesson 1 have a focus for catching up on what they missed. During the quiz, attendance can be checked and those students who have already been placed can be given their attendance sheets and course goals. After the quizzes are checked, they are signed by students to indicate they have read and understood their responsibilities.
The placement process makes Lesson 2 by far the busiest week for the teacher. The approach to placement within DynEd has two steps: a general Placement Test and the more specific Mastery Test. After coaching on its main features from the teacher, students take the Placement Test. The software will offer several possible courses depending on the student's score. Since courses differ in content and language, some courses (even at a similar level) may be more difficult or complex than others for the student. Once a course is selected, a student takes a Mastery Test to confirm a given level within that course is appropriate, and (if all is well) can then begin.
Although much of the broader process is automatic, some experience is required when assigning courses. Subtle nuances differentiate many courses, and inexperienced teachers may select a course that is a bit too easy or difficult for the student. This is not a problem if it is caught early on, but can lead to complications if the student doesn't say anything and it is overlooked. Teachers are therefore encouraged to talk with all students in the following classes in order to check placement.
As the placement process is done one-on-one after testing, it requires some patience on everyone's part. The result, however, is that students are placed within a course that is sufficiently stretching to be challenging, but not too easy or too hard. Students are given a target time to achieve by the following class, and asked to begin their course of study.

Subsequent classes
Once students begin their course of study, it becomes the teacher's responsibility to make sure that students understand what is expected of them in terms of study performance, but the students' responsibility to do the work to fulfill that expectation. There are many details that go into assessing successful study (e.g. study frequency, overall time, success in comprehension questions, etc.). With experience, teachers can easily recognize the most common problems and pitfalls that transpire during a student's course of study, and address those problems early.
Students are largely left to their own devices during class to achieve their course goals, with mini-coaching sessions as often as class size allows. The "Intelligent Tutor" feature is easy to use and offers comprehensive advice on study practice, but experience has shown that many students do not make use of this feature unless prompted to do so by the teacher. The purpose of these mini-coaching sessions is to review independent study habits, often with reference to the Intelligent Tutor, and offer advice about how students might study more effectively. These sessions help students interpret data and act on advice or information given.
Students are required to study independently outside of their scheduled class time. This "free study" (or self-study) outside of class is done with no teacher to offer assistance. The coaching sessions in class help to scaffold a student's ability to do this. The software requires multiple study sessions every week, and this requirement to study regularly is one of the keys to success in using the software, and in becoming comfortable with the language.
The software is accessible only on campus, so students must complete their free study at school. Students may do their free study at lunchtime or in any of the other repeaters' classes using DynEd, so long as they ask permission from the teacher of that class and as space allows. Computer rooms are open for student use at specific times during the week. Language Centre teachers usually monitor the lunch time study sessions during the first and last two weeks of the semester to help with any questions that students might have.

Passing the course
Once a student has completed 80% of a unit or module, he talks to the teacher about taking the Mastery Test for that section. Teacher and student will discuss the student record, and if it seems the student is ready, the test will be unlocked by the teacher and taken immediately in class. By default, the software is set to open tests for students automatically when they have achieved a certain level of study. This default setting was removed after problems occurred with students not being prepared for tests when they opened, and, in some cases, cheating (e.g. using notes or having friends take tests outside of class).
In the event students fail a test, they must show evidence of review and improvement, which can be obtained through the software's record system. Once a student and teacher agree that a different outcome from last time is possible, the students can retake the test. If, on the other hand, students complete units and pass tests before completing the necessary study time, alternative assignments are discussed.
A checklist provides evidence that a student has fulfilled all the criteria for the course. Both student and teacher can see that the class requirements have been met and a grade can be immediately assigned by the teacher based on those requirements. An exit questionnaire is then completed online and both student and teacher sign to show completion of the course and the final grade.

The supervision process
With some classes numbering in the forties, one teacher has described what he does as "educational triage". That is to say, going from student to student, trying to patch any problems as effectively and quickly as possible before moving on to the next, in order to help the largest number of people possible. The following is a discussion of the tools teachers use to aid the supervision process in order to help students reach their study goals.

Multiple class students
The number of students who fail repeaters' classes has been and remains high. Monitoring exact numbers has proved impossible, owing to several changes in record keeping (discussed later), but this is especially the case with students who are repeating more than one credit in the same semester. The work required to successfully complete a DynEd class is achievable, but demanding. Students who elect to study with DynEd are therefore advised not to take more than two such classes in any given semester. In the past, students registered for class face-to-face with a registration counselor, who would make clear the requirements for the classes. Students were personally told the difficulties involved with taking more than one DynEd class. In contrast, registration is now online. There is a stated warning on the registration web page about the challenges involved with simultaneously taking multiple DynEd classes, but many students choose to take three, four, or even five of these classes in a single semester. The students eventually find the workload to be too much and often fail some or all courses. This has been one of the biggest complications in terms of student success.
In the past, students taking multiple classes would find it hard to fit in the appropriate amount of free study time for each class. As a result, they felt unable to complete all (or any) of the classes. To counter this problem, the Language Centre teachers developed a system that allows students who are taking multiple classes with different teachers to study one class at a time for shorter periods. Under this system, students study the material for each single class in succession -rather than studying all classes simultaneously -until the requirements for a particular class have been met. Once a class is safely completed, they begin work on the next class, and so on. Although it is still rare for students taking a high number of classes to pass all of them, we are achieving more success with such students under this system. This system, however, requires extensive communication so that teachers who share a student taking multiple classes are on the same page regarding the student's study schedule.

Study sheets
Keeping track of student progress is vital, since success in class depends on consistent and progressive study. While regular feedback on progress is available from the Study Score and the Intelligent Tutor, some students (as mentioned earlier) often fail to check, comprehend, or act on the data they receive. Study sheets were therefore designed to help students keep track of their progress.
The sheets are simply graphic representations of what students are studying. Using the software, students consult their study records, then fill-in bar graphs on the study sheet to chart their progress through the various sections of the course they are studying. This gives them a visual mapping of their study progress. Interacting with the data this way lets them see which sections of the course have been completed, which sections need more studying, and when they are ready for the test. The sheet acts as a prompt to remind them to access the software's management system, and makes that data a common point of reference for the teacher and student. Over time, they develop a habit of using the software to check their progress. Instilling a sense of confidence through the establishment of clear goals and expectations as well as an increase in personal accountability helps to boost student motivation (Marimuthu& Soon, 2005), and the study sheets provide a way to make sure that happens.
Study sheets also serve as an exit form, which gives students a checklist of what must be done in order to pass the class. Since the software does not have an attendance or grading feature, the sheets are also used as a record of the students' attendance and final grades. For students taking multiple classes, study sheets help to show projected study paths for all classes, record which class they are working on, and indicate roughly when they are expected to finish each class.

Attendance
One of the main reasons for using the software is that it provides measurable evidence of work done and achievements made. For this reason, students who have a problem attending because of job searches or other reasons can make up what is missed, and be clearly shown to have completed a required course of study similar to any of their classmates. University policy states that if a student misses four or more classes, he will fail the class, butbecause work on DynEd is so measurable, students missing class for specific activities can be afforded more flexibility.
Students who do miss more than four classes generally fail due to lack of study time or overall poor study performance. For students in DynEd classes, chronic non-attendance continues to be a problem, sometimes with up to 40% of a class failing due to absences. As a result, a concerted effort is being made by the teachers to notify all students who are falling behind in their work due to absences. This is done weekly via an electronic bulletin board system that all students at the university are supposed to check daily. Although the effectiveness of this system remains unclear, many teachers report students with serial absences attending class after such notices have been delivered. It is seen as opening the door to a fresh beginning for students who would otherwise fail, and students can negotiate concrete steps to make up for what they have missed in an equitable manner.

Grade Point Average (GPA)
Owing to a new grading system, a higher number of students are failing the classes. Previously, non-attending students were dropped from the roll sheets, creating an artificially higher pass rate. This is no longer the case. Therefore, any comparisons of pass rates will be false. Also, owing to the curriculum change, the content of previous repeaters' classes cannot be compared.

Study time
Study time is recorded by the software. If no activity is recorded for a certain duration (approximately one minute), the system ceases to record the study time. Students who are not actively using the software for long periods of time are surprised to find this reflected in the time or unit completion levels recorded by the computer. Some students have attempted to "game the system" by randomly clicking buttons, or opening the recorder and not recording anything. This behavior is consistent with observations by Gobel (2008). In theory, while such behavior may "pad" recorded times and Study Score, it rarely helps acquisition, and often comes to light during the coaching sessions and in test scores.
One real problem affecting study time occurred when a few students reported losing small amounts of time on their study records. The length of time reported was seldom long, and rarely affected the same student twice. At first, teachers thought the loss of time was due to inactive study, as described above. On a couple of occasions, however, teachers were able to witness this loss of time first-hand, proving there was a technical problem. The origin of this problem has so far not been traced, although superficial evidence points to a system conflict within the school hard-or software. There is no evidence to show how student motivation was affected, but it is hoped that since the situation was dealt with quickly, students consider the system to be open and fair. Measures have since been put in place in case this happens again. On the (rare) occasions this happens, the loss is recorded with an open mind, and the time lost may be considered in relation to other factors of a student's performance, such as having improved comprehension levels, or simply being able to pass the test for the unit being studied.

Anti-cramming measures
In the academic year of 2011, some changes were introduced to the DynEd software midsemester, ostensibly to prevent cramming. The change was such that progress was not recorded after a student completes 20% of a given section in any given 24 hours. This was so that students would study several sections of a unit or module contiguously, rather than "cramming" one specific section.
At first, it was unclear whether or not study time was affected if students surpassed the 20% limit, as several students reported missing time during this period. Closer inspection revealed that originally the system was recording above 20% during 24 hours, but removing the percentage later (although this no longer happens). Students noticing this believed they had lost time, and it was sometime before teachers adapted to this change. The change itself is generally considered by teachers to be a positive one, but adapting to it and working out exactly how it affected students and teachers mid-semester took some effort.

The future
At present, the DynEd classes focus mainly on low-level students who have been unable to complete regular classes. Efforts are being made to expand the use of the software in order to take advantage of some of its untapped features. One problem that can plague classes that incorporate CALL is the lack of opportunities for students to engage in face-toface interaction (see Campbell, Brown, & Weatherford, 2008). This is exactly the case with the current DynEd classes. Students interact almost exclusively with the software. While some teachers do provide for limited teacher-student face-to-face interaction (mostly in the form of simple Q&A), there is almost no face-to-face interaction among students. Although the software is designed to blend with communicative learning, the disparate levels and studypaths in the current set-up do not promote pair-or group-work. At this time, the repeaters' classes function in a way that will allow the maximum number of students at various levels of English ability to be assembled in the fewest number of time slots.
In a move to better use the software, several teachers from regular classes -rather than repeaters' classes -are now using the software in blended programs that allow for more communicative activities. These classes have common content, and face-to-face work based on that content. It is hoped that these teachers will be able to continue the repeaters' classes should current teachers of those classes leave.
Successful advancement of the DynEd classes will require the dedication of a core group of teachers who share a desire to see the software utilized in as many ways as possible. As mentioned earlier, two of the original fulltime teachers involved have since left the school. Future losses (due mainly to contract limitations) will have a more profound effect, as administrative duties are split among the remaining teachers. Replacing them will require training not only in those duties, but also in the way the DynEd courseware functions. Consensus building must take place, especially when modifications need to be made, and this may be difficult with limited experience. It is quite possible that within the next two years all current Language Centre teachers will be gone, leaving the future of DynEd use at the school up in the air.

conclusion
In the two years since the repeaters' classes began full operationusing DynEd, teachers have developed a core of skills and techniques relating to the pedagogy of the classes and the management skills required. Teachers, both part-time and full-time, have put in many additional hours in order to build a working relationship with each other and the students. Events within the university have impacted the running of the course in ways too numerous to count, making any substantial measures of success very difficult to gather. In short, it has been hard. The system does seem to be functioning well as an ongoing process. Unfortunately, some substantial questions about how the positive progress and institutional learning will be maintained remain unanswered. What has been shown, however, is that with good teamwork and a pedagogically sound software solution, a good system can be implemented successfully.