An attempt to raise Japanese EFL learners ’ pragmatic awareness using online discourse completion tasks

Graduate School, Hokkai-Gakuen University nanaho.oki@gmail.com This practical paper discusses the effect of explicit instruction to raise Japanese efl learners’ pragmatic awareness using online discourse completion tasks. The five-part tasks developed by the authors use American tv drama scenes depicting particular speech acts and include explicit instruction in these speech acts. 46 Japanese efl college students enrolled in an English grammar course received explicit instruction and completed the tasks in five consecutive classes using an e-portfolio system that allows them to share their learning outcomes. Their pragmatic awareness was measured by preand post-instruction pragmatic judgment tasks. The results showed that the participants’ pragmatic awareness developed significantly after the instruction. The learners evaluated the awareness raising activities as effective; however, their learning outcomes reveal challenges in developing grammatical and lexical knowledge together with pragmatic knowledge. This paper will discuss the effects of the instruction, the participants’ perception of the usefulness of the tasks, and their actual responses to the tasks to further identify the challenges that Japanese efl learners should meet to develop well-balanced communicative competence in the language.


introduction
Pragmatic competence is defined as "the knowledge that influences and constrains speakers' choices regarding use of language in socially appropriate ways" (LoCastro, 2012, p. 307) and is regarded as a major component of an influential language knowledge model (Bachman & Palmer, 2010).It is also related to the overall objective of English in the Course of Study (mext, 2009) "to develop students' communication abilities such as… appropriately conveying information" (p. 1).l2 pragmatics or interlanguage pragmatics, "the study of how l2 learners learn how-to-say-what-to-whom-when" (Bardovi-Harlig, 2013, p. 216), has been widely studied for these past three decades as it has been claimed that l2 learners often develop grammatical competence in the absence of concomitant pragmatic competence (Bardovi-Harlig & Dörnyei, 1998, p. 233).
Previous studies have shown that pragmatic competence is both teachable and learnable for both efl and esl learners (Martinez-Flor & Fukuya, 2005;Olshtain & Cohen, 1990;Wigglesworth & Yates, 2007).There are also studies exploring the effect of instruction on Japanese efl learners' pragmatic competence (Kondo, 2008;Takahashi, 2001;Takimoto, 2008).In general, these studies found that explicit instruction had a stronger effect on learners' pragmatic competence than implicit instruction, for Japanese efl learners as well as for learners in different contexts.
While previous studies support the learnability of pragmatic competence, it is also claimed that there is "a noticeable gap between what research in pragmatics has found and how language is generally taught today" (Cohen, 2012, p. 33); in other words, pragmatics instruction in the language classroom is not taking place in an appropriate timing or manner.Sykes (2009) lists eight reasons to explain why this lack in language classrooms occurs: 1. limited theoretical support in course design and material creation, 2. lack of authentic curricular materials and appropriate input, 3. lack of instructor expertise, 4. a focus on microlevel features instead of macrolevel competence, 5. limited time available in the l2 classroom, 6. individual personality differences and sensitivity to certain factors influencing the interaction, 7. assessment and feedback challenges, and 8. immense dialectical, social and individual variation (pp. 203-204).
Pragmatic awareness, "the conscious, reflective, explicit knowledge about pragmatics" (Alcón & Jordà, 2008, p. 193), is crucial for the intercultural speaker to be competent at the pragmatic level (Cenoz, 2007) and plays an important role in developing pragmatic competence (Tagashira, Yamato, & Isoda, 2011).There are numerous studies that investigated relationships between pragmatic awareness and other related factors.Bardovi-Harlig and Dörnyei (1998) were some of the first researchers to examine the effects of learning environment on the development of pragmatic awareness along with grammatical awareness.Researchers have followed this line of research, investigating the development of l2 pragmatic and grammatical awareness in relation to various factors such as learning environment (Niezgoda & Röver, 2001;Schauer, 2006), proficiency (Niezgoda & Röver, 2001;Xu, Case, & Wang, 2009) and motivational profile (Tagashira et al., 2011;Yamato, Tagashira, & Isoda, 2013).Research has revealed that it is difficult for learners in foreign language learning settings to develop pragmatic awareness equivalent to that of esl learners (Schauer, 2006).
In order to fill the gap between what research has found regarding the importance of developing pragmatic competence and the lack of its teaching in Japanese efl settings, this practical study tackles the issue of raising pragmatic awareness with online discourse completion tasks using authentic video materials combined with explicit instruction by classroom instructors and the sharing of learning outcomes among peers on an e-portfolio system.The research questions are as follows: RQ1: Does explicit instruction using online discourse completion tasks raise Japanese efl learners' pragmatic awareness?
RQ2: How do Japanese efl learners respond to online discourse completion tasks combined with explicit instruction?
RQ3: How do Japanese efl learners perceive the effect of explicit instruction of pragmatic awareness?

Participants
The participants were 46 Japanese university freshmen majoring in English language and its cultures who were enrolled in a 15-week elective English grammar course to learn basic grammatical knowledge and skills.Each week, a ninety-minute class was conducted in a computer room allowing each participant to access online materials and tests.Demographic data of the same participants showed that there was no one who had spent and/or studied more than six months in an English speaking country.Judging from the passing marks for admission in the National Center Test for University Admissions, a standardized university entrance exam for public and private colleges, the participants' department is considered to be at an average academic level in Japan.

Online discourse completion task.
To provide video materials and to share the learning outcomes among the participants, the authors adopted Lexinote (Tanaka, Yonesaka, Ueno & Ohnishi, 2015), an e-portfolio system, as the platform for the online discourse completions tasks.Lexinote allows learners to record and save the target vocabulary items they encounter online, to search for them in online dictionaries, to practice them in several ways including written and oral rehearsals according to word familiarity (i.e., level of understanding of each lexical item), and to share their own learning experiences -in this study, responses to a discourse completion task -with peers.Each online discourse completion task consists of one screen with five parts.One part displays a discourse completion task that requires the participants to complete a speech act in writing such as refusal, response to compliment, and offer.Next is a video clip from an American tv drama series, The Big Bang Theory (cbs Corporation), in which the target speech act is depicted.Underneath the video clip is the input field for the participants to provide their own response to the task shown in the first part.The speech act performances included not only successful speech acts but also problematic ones in order to explain why certain types of performance may cause communication breakdowns.There is also the script of the video clip followed by an l1 explanation of typical and/or problematic speech acts and strategies.Finally, the participants' responses are shown in a display to allow them to share learning outcomes in class.Figure 1 shows an example screenshot of a refusal task for this study.

Pragmatic judgment task.
To measure the participants' pragmatic awareness, the authors adopted the pragmatic and grammatical judgment task developed by Bardovi-Harlig and Dörnyei (1998).The task is intended to reveal learners' awareness in two steps; first, learners identify whether an item is erroneous, either pragmatically or grammatically, or not; and second, if erroneous, learners judge how severe the error is on a 6-point Likert scale.The task contains 20 scenarios that include four speech acts: requests, apologies, suggestions, and refusals.
In order to avoid any practice effect of using the same task for both pre-and post-tests, the authors made two versions of the task using parallel sentences to the original task.Another revision was made for the number of items in the task to yield a wider spread in the scores.The original task consisted of eight grammatically incorrect items, eight pragmatically inappropriate items and four non-erroneous items.By adding three grammatically incorrect items, three pragmatically inappropriate items, and one non-erroneous item for each test, the authors developed two revised versions of the pragmatic and grammatical judgment task with a total of 27 items each.The newly added items were first reviewed by four English native speakers (one American and three Canadians), and then revised and reviewed by three native English speaking efl professionals in Japan, (one American, one Canadian, and one Australian).The final versions of the pragmatic and grammatical judgment tasks consisted of 11 grammatically incorrect items, 11 pragmatically inappropriate items and five non-erroneous items each.
Following up on other researchers' use of the original task (Tagashira et al, 2011), the authors also changed the choices to make it possible to see which types of errors the participants actually identify.The following is an example of pragmatic errors in the pretest (Set A) and the post-test (Set B).Task instructions were translated from English into Japanese by the authors.

Procedure
During a 15-week format semester, the pre-test was conducted in Week 3, five weeks before the instruction started, in order to assess the participants' initial pragmatic awareness as well as to avoid a practice effect by delivering two similar tests within a short period.The instruction started in Week 9 and lasted until Week 13.The post-test was conducted in Week 14.Both the pre-and post-tests were delivered via a learning management system, Glexa (Version2 Inc., 2013), at the participants' university.Table 1 shows the schedule of the tests and the instruction of speech acts.Each instruction session followed the procedures below.To raise participants' pragmatic awareness, throughout all the procedures, the instructors of the course, also the authors of this study, encouraged the participants to think of their own way of performing speech acts in English.Before showing a video clip, the instructors drew the participants' attention to the targeted speech act and to whether it was an ideal one or a failure, by explaining the background of the scene and the characters.After showing the video, the instructors explained various possible speech act strategies (e.g., Beebe et al., 1990) so that the participants could consider which strategies they should take given the context.Following the explanation, the participants responded to the online written discourse completion task.Since the responses sent to the system were automatically updated every 10 seconds, the participants were able to see other participants' responses immediately after they were sent to the system.After all the participants sent their responses, the instructors gave feedback regarding the strategies they had used and any grammatical mistakes they had made.According to the feedback they received, the participants deleted their initial responses and sent new responses if they wanted.

Pragmatic awareness
Only the pragmatic items in the judgment task were used in this analysis since the focus of this practical study is on pragmatic awareness.Because a total of eight participants were absent at either the pre-test or the post-test session, the data from 38 participants were used for the analysis.Table 2 shows the descriptive statistics of the scores of the pre-test and the post text.The results showed the effects of explicit instruction on participants' pragmatic awareness.Mean scores, with a maximum of 11 points, increased from 5.18 (sd = 2.42) in the pre-test to 6.13 (sd = 2.27) in the post-test; thus, the error recognition rate increased from 47% to 56% after instruction.A paired t-test between the pre-and post-test scores showed that there was a significant difference in the pragmatic error recognition rate between pre-and post-instruction with a medium effect size (t (37) = 2.71, p = .01,r = .41).The 95% confidence interval for the difference was from −1.66 to −0.24. Figure 2 shows the boxplots of the pre-test and the post-test.As for severity rating, i.e., how severely the participants evaluated the pragmatic errors, the scores increased slightly from the pre-test (M = 4.61, sd = 0.52) to the post-test (M = 4.79, sd = 0.73).Because it was not always the case that the same participants recognized the same types of errors in both tests, and also because the number of the participants who successfully recognized each error varied, the authors did not proceed with any further statistic analysis for severity rating.Whether classroom instruction affects learners' evaluation of pragmatic errors should be further investigated in a rigidly designed experimental study, because not only classroom instructions affect the evaluation of appropriateness of language use but also other factors such as individual and socio-cultural differences do so.

Learners' response to online discourse completion task
From among the four speech acts instructed in the study, this section focuses on one speech act, refusal, to see how the participants actually responded to the online discourse completion task and explicit instruction.The speech act of refusal has been claimed to be a major cross-cultural "sticking point" for non-native speakers and is interesting from a sociolinguistic point of view because of its complexity and its various forms (Beebe, Takahashi, & Uliss-Weltz, 1990).
Figure 3 shows the online discourse completion task of refusal to a request for Week 13 which requires the participants to refuse a friend's request to borrow their notebook.According to Beebe et al. (1990), the discourse of refusal by American English speakers is generally divided into three stages: give a positive opinion, make a statement of regret, and give an excuse.The following are some typical examples from the participants' responses.Any grammatical and/or vocabulary mistakes are left uncorrected.

One of your friends missed an important class and wants to borrow your notebook, but you have to refuse your friend's request for some reason. What would you say?
Your friend: Hi, do you have a minute?You: Sure.Your friend: I actually missed the economics class yesterday.I got up late, and I couldn't make it.But we have the midterm next week, right?So, I was wondering if I could borrow your notebook, I mean, just the last part is fine.You: Well, ( ).Your friend: Oh, OK.Don't worry.I'll ask some other friends.Thanks!The first example begins with empathy, "Well, that's too bad," and then a positive opinion, "I want (to) lend you my notebook," followed by an apology, "I'm sorry", and ending with an excuse, "I have to study hard today."Though there is a grammatical mistake, Student A clearly tried to follow the strategies as instructed.The second example begins with an apology, "I'm sorry" and then expresses regret "I wish I could [do] that," provides an excuse "I want to study at home," and makes a suggestion "I want you to ask (for) others."The use of "I'm sorry" here could be seen as a transfer of l1 use of "gomen(ne)" in Japanese, showing that Student B intends to be polite to the hearer, but this politeness is contradicted by the use of the impolite expression "I want you to" at the end.The third example begins with a statement of empathy, "I know how you feel," though it is difficult to judge whether the expression is appropriate or not in this situation considering individual differences in the perception of relationships with the friend.After that, Student C misuses the verb "borrow" instead of "lend" or "let you use" and uses the modal "could" instead of "should" or "would need to".Another possible inappropriate use is "You should have told me earlier", although it depends on how close the friends are.
Overall, these examples demonstrate that the participants tried to some extent to follow the strategies or the discourse of refusal, as instructed in the class.However, many of the overall responses contain grammatical and/or word usage errors that affect the whole discourse and the hearer's impression of the speaker.Also, it should be noted that these responses were collected immediately after each instructional session, but whether the participants actually acquired a particular pragmatic strategy would have to be further investigated using a delayed post-test, which was not included in this study.

Learners' perception of explicit instruction of pragmatic awareness
Regarding questionnaire Item 1, the usefulness of Lexinote in learning the appropriate use of English, the results showed that the participants perceived the online task to be useful to some extent (M = 4.70, sd = 0.90).Although it is difficult to tell whether this mean score is meaningful or not without additional survey items, only three participants responded "I don't agree" or "If I have to choose, I don't agree.",and no participants chose "I don't agree at all." The participants' responses to open-ended Item 2 asking what they thought of the discourse completion task using Lexinote, showed a positive evaluation of the instruction and of the task, and also implied how these should be modified.Below are three excerpts from the three negative respondents, followed by three excerpts from the other 43 positive respondents.The negative responses were related either to uncertainty about using Lexinote or to doubts of using it compared to face-to-face practice.The typical positive responses were related to the following three points: the effectiveness of observing conversation in a tv drama, reflections on their own language use, and the advantage of sharing learning products with peers.All the excerpts below were translated from Japanese into English by the authors.

conclusion
This practical study showed that the explicit instruction using discourse completion tasks had a significant impact on Japanese efl learners' pragmatic awareness, and that the learners perceived the usefulness of the tasks in learning appropriate use of English.It also showed that the learners in this study attempted to construct their discourse using the strategies that were taught in class despite challenges in grammatical and word usage knowledge.
The fact that learners have difficulties in both correctness and appropriateness of language use implies that Japanese efl instructors should conduct classes that balance grammatical and pragmatic aspects of language.It is claimed, however, that "textbooks still tend to provide only limited coverage of how language in context conveys messages directly and indirectly" (Cohen, 2012, p. 34).Classroom instructors' efforts must necessarily play a significant role in developing learners' well-balanced language competence.
In order to further investigate the potential of explicit instruction on Japanese efl learners' l2 pragmatic awareness in a rigidly designed experimental study, the following limitations have to be solved.First, as the instruction of this study was done in only five classes using less than 30 minutes each, it is necessary to examine the effect of instruction over a longer period and to examine whether the effect will last over time in a pre-test, post-test and delayed post-test design.Second, though the authors carefully used parallel sentences in similar situations to the original judgment task by Bardovi-Harlig & Dörnyei (1998), it is also necessary to confirm that the two re-designed awareness tasks in this study are at the same level of difficulty.Finally, in order to see whether learners actually benefit from the instruction, a future study should use an oral discourse completion task to elicit learners' actual language use.Since this study employed a written online discourse completion task where the participants could spend some time before they responded to the task, the results would have been different if oral discourse completion tasks had been used.By solving these limitations, researchers and classroom instructors can further reveal the potential of explicit instruction on l2 learners' pragmatic awareness.

acknowledgments
This study was supported by a Grant-in-Aid for Scientific Research (kakenhi) (C), (No. 25370646).The authors would like to thank Suzanne Yonesaka at Hokkai-Gakuen University for her insightful comments on the manuscript.Their gratitude also extends to the anonymous reviewers for their comments on the study.
1. Participants watch a video clip depicting a particular speech act.(3 min.)2. Participants listen to and read the teacher's l1 explanation of pragmatic strategies and cross-cultural differences.(10 min.)3. Participants respond to the online written discourse completion task.(3-5 min.)4. Participants share their responses on the screen and the instructor comments on their work.(5-10 min.)

Figure 2 .
Figure 2. Boxplots of the pre-test and the post-test results.

Figure 3 .
Figure 3. Discourse completion task for Week 13: Refusal to request.

An example question of the revised pragmatic and grammatical judgment task (prag- matic error): Set A
It's Anna's day to give her talk in class, but she's not ready.Teacher: Thank you Peter, that was very interesting.Anna, it's your turn to give your talk.

An example question of the revised pragmatic and grammatical judgment task (prag- matic error
In order to see how Japanese efl learners perceive the effect of the explicit instruction, i.e., research question 3, the authors used a simple two-item questionnaire.Item 1 uses a 6-point Likert scale from "1.I do not agree at all." to "6.I strongly agree."Item 2 is an open-ended question asking the participants to state their perception of the online completion tasks.1. Class assignments using Lexinote are useful to learn appropriate use of English.2. What do you think of the discourse completion task using Lexinote? ): Set B It's Anna's day to give her presentation in class, but she's not ready.Teacher: Thank you Daniel, that was really amazing.Anna, it's your turn to give your presentation.(!) Anna: I can't do it today, but I will do it tomorrow.The last part (!) is … 1. grammatically incorrect 2. inappropriate for the situation 3. grammatically correct and appropriate for the situation 4. I don't know If there is a problem, how bad do you think it is?Not bad at all___:___:___:___:___:___Very bad 2.2.3 Questionnaire.

Table 1 :
The tests and speech act instruction schedule

Table 2 :
Descriptive statistics of the pre-test and the post-test(n = 38)