A corpus-based system of error detection and revision suggestion for Spanish learners in Taiwan : A case study

Cheng-Yu Chang National Cheng Kung University, Taiwan ccy0927@gmail.com Compared with English learners, Spanish learners have fewer resources for automatic error detection and revision and following the current integrative Computer Assisted Language Learning (CALL), we combined corpus-based approach and CALL to create the System of Error Detection and Revision Suggestion (SEDRS) for learning Spanish. !rough corpus-based data training and related applications, this system was designed specially for learners of Spanish in Taiwan. !e Corpus of Taiwanese Learners of Spanish (CTLS) was used as a database to facilitate the development of the system. !e learners’ corpus was tagged with part-of-speech (POS) and lemma information, and it was also annotated by native Spanish speakers with revisions corresponding to errors made by students in their original texts. !e system can, in real time, identify tri-gram errors based on training data extracted from the revised texts of the learners’ corpus and provide revision suggestions listed according to usage frequency for users. !e system was evaluated by "# Spanish learners and eight experienced programmers to quantify the system’s practical effectiveness. In addition, feedback from learners’ was collected to improve the system in the future.

Compared with English learners, Spanish learners have fewer resources for automatic error detection and revision and following the current integrative Computer Assisted Language Learning (CALL), we combined corpus-based approach and CALL to create the System of Error Detection and Revision Suggestion (SEDRS) for learning Spanish.
rough corpus-based data training and related applications, this system was designed specially for learners of Spanish in Taiwan.
e Corpus of Taiwanese Learners of Spanish (CTLS) was used as a database to facilitate the development of the system.e learners' corpus was tagged with part-of-speech (POS) and lemma information, and it was also annotated by native Spanish speakers with revisions corresponding to errors made by students in their original texts.e system can, in real time, identify tri-gram errors based on training data extracted from the revised texts of the learners' corpus and provide revision suggestions listed according to usage frequency for users.e system was evaluated by Spanish learners and eight experienced programmers to quantify the system's practical effectiveness.In addition, feedback from learners' was collected to improve the system in the future.

Introduction
Computer-assisted language learning (CALL) is defined as "applications of the computer in language teaching and learning" (Levy, ), and various studies have developed this concept since the s, going through three main phases commonly termed behavioral, communicative, and integrative CALL (Warschauer, ).In early behavioral CALL, computers provided teaching and practice materials, serving as drill tools for learners.Later, in the communicative phase, computers stimulated learners' thinking and allowed them to discover the answer or results independently.In the current phase known as integrative CALL, learners can, through the medium of computers, utilize multimedia, software, the Internet, and various corpora and concordances to facilitate their language learning of different linguistic aspects such as vocabulary, phrases, grammar, and collocation.In addition, corpora and related tools have played an important role in different sub-areas of linguistics because of the convenience they provide in analyzing data and reaching generalized results (Biber, Conrad, & Reppen, ).Because of a lack of available corpus-based error detection and revision suggestion tools, especially for Taiwanese learners of Spanish, by applying integrative CALL with a constructed learners' corpus, we created a preliminary tool called the System of Error Detection and Revision Suggestions (SEDRS) for Spanish learners.e aim of this tool is to help learners of Spanish gain lexical and grammatical knowledge of the target language.In Section of this study, we review the relevant literature regarding CALL and corpus linguistics.In Section , we present the development of the SEDRS system and its implementation for error detection and revision suggestions for Spanish language learning.We outline the evaluation by experts and users as well as the questionnaire about the practical effectiveness of the SEDRS system in Section .In Section , we discuss the limitations of our findings and propose further refinements and future developments for the SEDRS system.We summarize the relevance and significance of our development and initial evaluation of the SEDRS system in Section .

Literature review
In speaking of the advantages of using computer-assisted tools to facilitate language learning, Hashemi and Aziznezhad ( ) point out that using CALL not only has the advantage of offering a powerful self-access facility to language learners, but also gives a new role to teaching materials.Learners can experience their individual learning styles, and also have more interaction with teaching materials than with a conventional teaching model.Learner-centeredness and interaction are two important factors, especially for the development of language production ability (Weimer, ).To develop writing ability, learners need to practice in a more autonomous learning environment; therefore, automatic computer-assisted tools for identifying errors and providing revision suggestions are required to satisfy individual needs in an efficient way and provide a different kind of interaction.
Meanwhile, over the last two decades, corpus-based learning approaches have drawn a lot of attention and have been adopted for various sub-areas of linguistics.With advanced knowledge of computational linguistics and integrated technology, a large amount of data can be analyzed efficiently, much faster than has ever been possible in the past.Data can be compiled for the construction of a specific corpus that can lead to conclusive generalizations in various fields of study.Without the integration of CALL, learners need to be familiar with the search functions and selection principles of corpus tools.erefore, following the current trend (Johns, Lee, & Wang, ; Granger, ) of the integrative CALL phase, we combined a corpus-based learning approach with CALL to develop a system for error detection and revision suggestions that is easily accessible for language learners.
Previous studies such as Geluso ( ) and Acar, Geluso and Shiki ( ) show that using Google as corpus, by typing ungrammatical strings in the search bar and correction will be provided directly.Besides, users can observe total results of the usage and make a revision according to frequency of Google search result.Furthermore, Wu, Witten and Franken ( ) and Shei ( ) use Google as web-based corpus to conduct search for language teaching and learning, they point out that learners get obvious progress on writing ability for collocation.Other related studies for English such as Chang, Chang, Chen and Liou ( ) indicates a corpus-based automatic collocation writing assisted system for Taiwanese EFL students by detecting and correcting errors in collocation especially for the combination of verb and noun.However, this study only emphasizes detecting and revising single unit and collocation errors.Besides, Levison, Lessard and Walker ( ) proposes an error analysis system which is able to detect lexical and morphological error for French writing and provides feedback for learners.
Research on error detection and revision for the Spanish language includes study by Bustamante and León ( ).GramCheck is considered the earliest grammar checker for Spanish (Bustamante & León, ).GramCheck can identify error types, including those related to gender concordance, prepositions, the passive voice, and gerunds, but it is not available for public use.Furthermore, recent resources with specific linguistic features include LoMás TV, Spanish Checker, Softpedia-Spanish ., the El Corrector Spanish Grammar and Spell Checker CD-ROM, Language Tool ., Free Online Spell Checker, and spellchecker.net.With regard to revising functionality, most of these tools, such as LoMás TV and Softpedia-Spanish ., are limited to revising only orthographical and morphological errors and lack the ability to correct higher-level grammatical errors; for example, those involving subject-verb agreement or syntactic structure.Among those that are commercially marketed, Spanish Checker uses a whole text editor and a private mode.
Although there are a variety of applications of CALL, the limitations of CALL research cannot be ignored (Chambers, ; Felix, ; Hubbard, ).Furthermore, computers are limited to solving only expected and preset problems and cannot substitute for teachers.
e goal of this study is to identify and correct as many learner errors as possible based on the training data.
After considering the reviewed literature we sought to integrate a learners' corpus and build an SEDRS for writers learning Spanish.We wanted to keep the design learner-centered and relevant to the learners' purpose.e corpus was therefore limited to text supplied by learners and revised by native Spanish speakers, using the revised learner materials as the source of data makes the corpus more closely related to our subject of study and more reliable for learning from a linguistic perspective than using a broader corpus such as that used by Google.With the goal of producing a computer assisted tool that detects learner errors and offers revision suggestions that span different linguistic levels in a format that can be made available for public use at no charge, the present study will answer the following research questions.
. What are the major functions of the developed corpus-based CALL system? .How do these functions work in assisting learners improve their writing?
. What are the advantages and disadvantages of the system?

Methodology
e method for conducting this research included two major parts, the development and then the evaluation of the system.

Developing the System of Error Detection and Revision Suggestion
To develop the SEDRS, we first considered the major models of intelligent computer-assisted learning systems, as described by Nyns ( ).From these, we selected a natural language interface, a pedagogical module, a model for analyzing students' errors and language acquisition and a knowledge base of the subject domain to serve as the basis for the SEDRS.In addition, we included functions for detecting and revising grammatical, lexical, and semantic errors based on the general category of error (Young et al., ).By doing so, we intended to cover general errors of different linguistic levels that Taiwanese learners of Spanish usually make.We also integrated a mechanism, as proposed by Kukich ( ), which would process language errors using the following three-step process: ( ) the location of the error is determined, ( ) the error is compared with an existing database based on statistical probability, and ( ) the error is corrected based on contextual information.
Figure shows the framework of the SEDRS system.e detection module identifies the dubious or incorrect segments that are present in the text.When potential errors are identified, the system searches for and displays corresponding suggestions.We used the Corpus of Taiwanese Learners of Spanish (CTLS ) to achieve the goals of our system.Revised texts from learner corpus were used to build a reference database for error detection and revision suggestion lists.A total of , texts and , words were extracted from the CTLS as training data for the development of the system.Afterward, part-of-speech (POS) and lemmas were tagged and three-word sequences (tri-gram) were segmented before the program was implemented.

CEATE POS-tagging
We collected training samples from revised texts, which were used by the computer to identify correct and incorrect data.e correct data established boundaries for comparing input texts.If input texts fell within the established boundaries, they were identified as correct by the system.If not, the errors were filtered.e implementation was based on asynchronous JavaScript and XML (AJAX), and tri-gram structure units were adopted; i.e., every tri-gram (A+B+C, B+C+D, C+D+E) with overlapping elements was examined one by one.When users input texts, the system checked every tri-gram and compared it with the reference database.If a potential error was detected, the tri-gram was underlined to notify the user.
e revision suggestions were also generated from the CTLS revised articles.e lists were sorted according to frequency.In implementation, we used a set of hotkeys to retrieve suggestions from the database and incorporated them into the user interface.e programming languages used for revision suggestions at runtime are PHP, AJAX, and MySQL, and the PHP programming language is used to read data in the MySQL database.When the data are loaded, the system uses AJAX to detect user input.
Since the training data was originally produced by Taiwanese learners of Spanish whose native language was Chinese, this tool for error detection and revision was designed especially for Chinese learners of Spanish and then were revised by native Spanish speakers.
e specific source of compiled data in the created corpus was chosen to tailor the system for Taiwanese learners of Spanish.

Evaluation
We conducted the following evaluation from two perspectives, experts and learners, so that the system can be improved in the future.
. .Expert evaluation.For feedback on technical aspects, we evaluated how users interacted with system.Lin et al. ( ) utilize a Purdue Usability Testing Questionnaire that contains criteria such as compatibility, consistency, minimal action and user guidance to evaluate human-computer interactions.We revised and translated questions into Chinese for our questionnaire (see Appendix A) and invited experienced programmers from the Department of Computer Science and Information Engineering to share their opinions.
e questionnaire contained nine questions.Experts provided response on a Likert scale ranging from to : ( ) Strongly disagree, ( ) Disagree, ( ) Neither agree nor disagree, ( ) Agree, ( ) Strongly agree.Not all criteria of the Purdue Usability Testing Questionnaire were included in the questionnaire; flexibility, for example, the ability to let users have a customized interface or to provide a feedback platform whenever users encountered problems is not included in SEDRS.We hope to include this feature in the future.Other criteria such as learnability (need for users to learn how to use the system), minimal memory load (need for users to remember abbreviations), and perceptual limitation (acceptance for items arrangement and color usage) were integrated these criteria into the nine questions in the questionnaire we gave to experts.

. . Evaluation by learners.
Participants and assessment design.To evaluate the practical effectiveness of the SEDRS from the user's perspective, we had twenty-five participants type sentences and then complete a questionnaire.Twenty-five Taiwanese students ( women and men between the ages of and ) from the Department of Foreign Languages of National Cheng Kung University in Taiwan participated in the evaluation.ey had previously completed two semesters of Spanish (approximately classroom hours) using Dos Mundos as the textbook.An explanation (see section .) and brief instruction on the use of the system were given before participants completed the test and questionnaire.Function test.We tested the error detection and revision suggestion functions of the SEDRS using sentences (listed in the first section of Appendix B) that contained orthographical, morphological or grammatical errors.e sentences were extracted randomly from the training data of the revised texts of the Corpus of Taiwanese Learners of Spanish.
Participants were asked to type in these sentences into the blank screen of the SEDRS (see Figure ).

Figure 2. Blank screen of SEDRS
First, participants were asked to keep track of errors that the system detected.For each error, students chose from the list of possible corrections suggested by the system.e section was limited to minutes.en, participants were given another minutes to type in an independent previously composed writing sample to test the two main functions of the system.e composition was typed in one sentence at a time.Errors in each sentence were detected and possible revisions from the suggestion list were selected before the next sentence was entered.Finally, the error detection and revision suggestion functions of the SEDRS were analyzed by researchers from participants' results.
Questionnaire.Students who participated in the functions tests were asked to complete a questionnaire to provide their opinion regarding user convenience of the SEDRS.e questionnaire included two subsections to assess user satisfaction regarding the instructions and the interface.Several open questions addressed the advantages and disadvantages of the system and elicited suggestions for future modifications.e first two sections addressed instruction assistance, functionality, and other details that were associated with (A) the instruction manual and (B) the interactive interface (Appendix C).Answers were provided on a Likert scale ranging from to : ( ) Strongly disagree, ( ) Disagree, ( ) Neither agree nor disagree, ( ) Agree, ( ) Strongly agree.e participants had minutes to finish the questionnaire.For the section of open questions, we calculated the number of participants who shared the similar opinions.

Instructions
e developed system uses a check-as-you-type function, which distinguishes it from other tools described in Section .e instructions are shown as follows.
Instructions: SEDRS uses a check-as-you-type function.
. Users type in the panel, and SEDRS detects the last three words to see if there is potential error. .When words are underlined in red: a. Users can press ↓ on keyboard to view the database.b. en, users can press → on the keyboard to check the suggestions list. .To return to the database, users press ←.
When a possible error occurs, the system responds immediately.e potential tri-gram errors, such as orthographical, morphological, grammatical, lexical and semantic errors, are underlined for notification (step ).Users can then use the keyboard arrows to select an appropriate revision from the list (step ).After users select a possible revision from the suggestion list, the item replaces the error (step ).

Evaluation
We will answer the research questions ( ) and ( ) through the result of evaluation task which is consisted of two parts, expert and user evaluation.
. .Expert evaluation.e evaluation results provided by experienced programmers are shown in Table .

Questions Mean
1 SEDRS is compatible with users' computer and software.
3.5 2 Updates can be loaded easily into the system.
4.375 3 Graphics and colors are used appropriately for instructions.
4.375 4 Operations of cursor switching and panel scrolling are smooth.
3.625 5 Users can easily install the system.
4.25 6 Users can easily start and end using the system.
4.5 7 e system provides minimal steps for manipulation.
4.25 8 e information is clear, concise, and informative to users.4.125 9 e organization is clear, logical, and effective, and easy to understand. 4 e questionnaire show that most of the experts agreed that the SEDRS performed well with an average score higher than , for the four criteria: compatibility (Q -Q ), consistency (Q -Q ), minimal action (Q -Q ) and user guidance (Q -Q ).Some of the experts were not satisfied with the compatibility because the operation of the system conflicts with Skype, which must be closed to operate the system.One expert indicated that the SEDRS could not be opened on a Linux operating system.Some experts suggested that the SEDRS should be mouse operated as well as keyboard operated.Experts reported the highest satisfaction in the areas of minimal action and user guidance, with scores above .

. . User evaluation.
Tested sentences.Initially, students were asked to type in sentences, however, the speed with which the tool was used to reach the revision goals varied among the participants.All participants finished revising sentences S -S within minutes, requiring approximately minutes for each sentence.However, only % of the participants finished all sentences and most were not able to finish within the time limit.Questions S -S were not completed by all the participants.erefore, we considered only the first ten (S -S ) sentences for our calculations and analysis.e results of our evaluation of the error detection and revision suggestion functions are presented in Table .Table shows distribution results of tested sentences.e system addressed sentence errors in two consecutive steps, error detection followed by revision suggestions.All the participants responded to underlined errors and selected a suggestion to correct the following error types in sentences S , S and S -S : orthographical (S ), reflexive pronoun-verb agreement (S ), gender agreement between an article and a noun (S ), plurality agreement between an adjective and a noun (S and S ) and preposition usage in a verbal phrase (S ).Among the sentences, the tool assisted over % of the participants in correctly revising errors, such as subject-verb agreement (S ) and verb usage (S -S ).us, the tool was useful for detecting and revising most types of linguistic errors.
e results of the free writing section showed the usefulness of the SEDRS to be limited.e system could successfully detect errors and provide proper suggestions to learners for the sentence contained in training data.e SEDRS requires improvement by enlarging the size of its database.

Questionnaire.
Satisfaction: e results of the questionnaire surveying users' satisfaction with the instruction manual and the interactive interface are shown in Table .e majority of the participants selected a satisfaction level of or higher with regard to the instruction manual (A -A ), which was more positive than the satisfaction expressed for the interactive interface (B -B ).More detailed information will be included in the  instruction manual in the future.e degree of satisfaction with the interactive interface varied between the different features of the system (B -B ).e participants were satisfied with the straightforward use of the functions (B ) and the ability of the tool to facilitate self-learning (B ).Participants expressed satisfaction with the appropriateness of the selections provided in the suggestion list (B ).However, the participants were more satisfied with the system's ability to detect orthographic errors (B ) than its ability to detect grammatical (B ) and lexical errors (B ).e low level of satisfaction that was expressed regarding the reaction times for detection (B ) and suggestions (B ) indicates that further technical adjustments are needed.us, the instruction manual is comprehensible, and the interface is generally user-friendly and useful for error detection and revision suggestion over a range of linguistic categories.
Advantages and disadvantages: Detection of errors.e advantages described by the participants in the open questions included the cross-check of spelling ( participants) and grammatical errors ( participants).ey also stated that the immediate detection of errors was advantageous because it reinforced their knowledge of grammar and usage, helping to decrease the number of subsequent errors ( participants).Due to the detection function of the SEDRS, students could immediately confirm what they had written.However, if the SEDRS underlined the writing, it reminded users that there may be potential error, and they would think twice before repeating the same error in subsequent writing.
e disadvantages that were cited included a problem caused by the method of detecting errors instead of analyzing a sentence as a complete unit ( participants).e evaluation results indicate that the training database was too small, which caused some incorrect detection of grammatical errors.e system lacked the necessary sensitivity ( participants).
Advantages and disadvantages: Revision suggestions.One advantage of the revision suggestion function reported in the questionnaires was the suggestion list for revising lexical and grammatical errors ( participants). is function was informative and offered a rich, broad range of selections to choose from ( participants).However, participants reported that the suggestions on the list did not always correspond to the original text ( participants), which is another indication that the database may not have been large enough to cover all possible revisions.No revision suggestions were provided for certain detected errors ( participant), and the most appropriate candidate was not always presented as the first choice in suggestion lists ( participant).is user feedback provides constructive suggestions for the future modification of the SEDRS.

Limitations
e limitations of the SEDRS primarily involve the tri-gram method of error detection and its inability to detect errors beyond the last three words that are typed.Every tri-gram needs to be examined and compared with the training data and the system cannot identify errors effectively in longer sentences or constructions that are more complex, including subject-verb inversion constructions. is was most evident in the free writing evaluation in which typed sentences were not limited to the training data.
e evaluations were limited by the number of participants.A more comprehensive evaluation with a greater number of participants (more than ) with a wider the range of proficiency levels and from other universities in Taiwan would provide more conclusive results.A comparison of two groups of users would be useful to verify the practical effectiveness of the developed tool.

Future work
A major objective for the future improvement of the SEDRS is to significantly increase the amount of training data. is should enhance the effectiveness of both the error detection and revision suggestion functions.We plan to expand the corpus of suggestion lists using Spanish Wikipedia to offer users more advanced information.Including a large-scale corpus may involve indexing technology including data compilation, parsing, and storage for a better information retrieval.We also plan to provide suggestion lists based on the results offered by Spanish collocations.Additional supplementary functions, such as a dictionary, will be added at the second stage for checking the entire texts to compensate for deficiencies associated with the tri-gram method of error detection.We will also expand our system to include a two-stage architecture.Users at the first stage can interact with the system in real time.After users finish the first stage, the system can check the entire article again for other errors, such as spelling, grammar, and verb tense.

Conclusion
We developed the SEDRS to assist learners of the Spanish language in Taiwan. is system differs from existing tools, particularly with regard to its check-as-you-type function.Based on the training results of the revised data comprised in the Corpus of Taiwanese Learners of Spanish, the primary construction is complete.Errors within three-word strings are identified, and a list of suggestions is provided by the system.Using these two major functions users can learn through writing by instantly detecting and correcting errors.is can help students avoid repeating similar mistakes.e SEDRS can be used as a tool for self-learning.
e evaluation and questionnaire results show that certain modifications are necessary to improve performance and efficiency of the SEDRS.e tri-gram reference corpus must be made robust enough to offer a wider range of error detection and revision suggestions.
e list of suggestions must be presented in an order appropriate to the context, and the reaction time for checking must be reasonable.e enlargement of the training database and the addition of other types of reference sources are also required.

Acknowledgement
is research was supported by a grant from e National Science Council of Taiwan (NSC --H---MY ).We wish to thank the CSIE team at National Cheng Kung University for their technical support, the students and instructors who volunteered for the CTLS and SEDRS, and the research assistants and native-Spanish speakers who contributed to this project.
e Corpus of Taiwanese Learners of Spanish (in Spanish, Corpus Escrito de Aprendices Taiwaneses de Español-CEATE) is comprised of , texts and , words since year and is a sub-corpus of the Multilingual corpora created by National Cheng Kung

Figure 3 .
Figure 3. Instruction manual and interface

Table 2 :
e results of function test

Table 3 :
Results of questionnaire