English Internet usage: An indicator of EFL progress

Seigakuin University j_byrne@seigakuin-univ.ac.jp The study attempts to prove, or disprove, the hypothesis that English Internet usage is an indicator of English progress among freshman at a Japanese university. The background to this question is the notion that students, who independently use the English domain of the Internet are highly motivated and gain fluency practice. However, it was not clear how many, if any, of the students would be using the English domain of the Internet. The evidence suggests that an unexpectedly high number of students are using the English Internet and they are, in simple mean terms, making significantly more progress than students who do not. Moreover, with the removal of the masking effect of the entertainment variable, the English domain students' mean average progress is double that of the full cohort.


Introduction
The Internet is reaching the significance of a social utility (Carr, 2005). Whilst, there are many linguistic domains and cultural subdomains, English plays a very broad role in the Internet eco-system (Crystal, 2001). Hence, there is an opportunity for many people around the world to use English. The recent and rapid introduction of Computer Mediated Communication (CMC) makes the opportunities more user-friendly, interesting and desirable. The fundamental question posed in this study is do Japanese university students use the English Internet? If so, what effect does it have on their English proficiency? The study took place within one university in Japan with a cross-section of freshmen students. It makes use of quantitative survey and English test score data. The primary concern of the study was to parameters, during my search of the literature with regards to my specific parameters, no correlation between Internet usage and English progress was found.
Historically, there has been little interest among EFL teachers in incorporating computers into EFL pedagogy (Olsen, 1980). However, that said, the societal paradigm has shifted once again with the introduction of mobile and communication driven technologies. Whereas in the 1980s and 1990s the emphasis in CALL was on how computers could be used to help people study English, in recent years, technology has been shifting the patterns of human communicative interaction on a worldwide scale. An increasing number of people require English to operate within the Internet environment. Crystal (2001) notes that the discussion on the Internet has shifted from technology to people, new forms of social interaction and purpose. It would appear language and communication have become central to this discussion.
Japan is at the forefront of consumer CMC technological innovation. According to Internet World Stats (2011), 78% of Japanese speakers are using the Internet. In particular, Japanese consumers have taken to mobile Internet enabled devices. According to Chinnery (2006) mobile smart-phone usage is near universal among Japanese college age students. In fact, Thornton and Houser (2005) found that during their survey of 333 Japanese university students, 100% reported owning a mobile phone. Furthermore, this social penetration of the Internet in Japan, combined with Japan's technological prowess and financial muscle, have meant this market is well catered to in the Japanese language; they simply do not need to use English.
However, we now live in an age of computer mediated communication, which may lead to change at the individual level. CMC has been made possible through the creation of software such as chat, e-mail and conferencing technologies. Blake (2008) makes a distinction between first and second-generation CMC tools. For example, e-mail and mailing lists are asynchronous first generation. The second-generation asynchronous tools include blogs and wikis. Synchronous CMC has shifted from typed chat to voice and video chat tools. Teachers have used these tools to create task-based activities. According to Warschauer (2004) this relates to the third stage of CALL development with the principle objective of <I>Agency. <I> Essentially, <I>agency<I> relates to using English and CALL to have a real impact on people's lives.
Against this backdrop, it was not assumed that Japanese students would be using the English domain of the Internet, but if they were, then it was suspected that such English Internet usage might have an effect on English progress.

Method
The hypothesis of this research lent itself to a focus on gathering quantitative data through a multi-method approach. This data gathering took the form of quantitative questionnaire survey data and quantitative student test result data. Firstly, a qualitative, open question, survey was used with a small group of student subjects, to assess the quality and likely performance of the main quantitative questionnaire survey. Secondly, student subjects were invited to provide additional qualitative data in a comment section at the end of the main questionnaire survey. However, the main thrust was quantitative. The questionnaire survey was designed to discover the extent of English domain Internet usage among the students and quantify that data. The student test data is useful for providing a statistical measure of student English proficiency and progress. The two sets of quantified data were then analysed for strength of correlation.

Data collection
The research took place at a small private Japanese university on the outskirts of Tokyo. An opportunity sample of 320 subjects was selected from the first year population. The first years were selected as they are independently (of the teacher) pretested and post-tested with a norm referenced test.
It appeared most sensible to survey all of the students present, on any given day, in one sitting per a given class. If a random sample from each class had been selected, there may have been side effects such as the Hawthorne effect (Brown, 2008).
Three departments were selected from the six departments at the University on the basis of opportunity and environmental similarity; Euro-American Cultural Studies, Economics & Political Science, and Local Community Policy Studies. The survey subjects in addition to coming from selected departments were also selected from the 645 registered students who had taken the Secondary Level English Proficiency (SLEP) pre-test. The final analysis only considered the students who had taken the pre-test, were available for the survey sitting, filled out the survey correctly and attended the post-test. Hence, the original sample of 320 subjects dropped to a much smaller working cohort of 121 subjects.

Ethics
The data collection raised a number of ethical questions and in keeping with social research standards, certain ethical procedures were needed to be followed. In terms of consent, the permission of the Director of the English Programme was obtained and the students were informed in both English and Japanese that the questionnaire survey was voluntary. Due to the need to link the survey data and the proficiency data, the research could not offer the students anonymity. Whilst this may not have an effect on the reliability of the data collected (Pearlin, 1961), it does raise concerns with regards to student confidentiality (Miller, 1971).

Survey design
The survey was encoded with a visible respondent identification code (Dickson et al., 1977;Grube, Morgan, & Kearney, 1989), providing a measure of anonymity should the respondent sheet be misplaced. The questionnaire employed several standard techniques and question types. Predominantly yes/no, but also categories, and Likert scale, questions were used. All of the questions were closed questions, although room was left on the back of the answer sheet for respondent comments. Dörnyei (2010) states that to ensure reliability, the survey should be internally consistent and that this is achieved using multi-item questions and a high Cronbach Alpha coefficient during item analysis. This research mainly uses single-item questions. In defence, the questionnaire questions mainly refer to habitual behaviour rather than subjective opinions or extracting emotional responses. Hence, the researcher believes that subjectivity may not be a large issue. The survey is primarily asking about the subjects' personal habits and thus the only knowledge that is required is whether they do or do not do something. However, given the generality of many of the questions, examples of what was meant to be conveyed were provided. The researcher does not believe that the questions are leading, presumptive or hypothetical. Cultural sensitivity could have been an issue, consequently a Japanese colleague was asked to review the questions and the pilot respondents were surveyed for their views in this respect.

Measuring progress
For this research and indeed the working definition at the University that the research was undertaken within, proficiency is defined as a numerical score on a SLEP test. Thus the proficiency progress has been defined for this research as an increase in score on the SLEP test over a ten-month period.
The Secondary Level English Proficiency (SLEP) test is an American test designed for ESL students within the American secondary school system. The test was designed as a norm referenced, high school version of the TOEFL test, that measures ability to understand written and spoken English (Foster, 1987). Consequently, it is not fully appropriate to the Japanese university context. However, there are compelling reasons for the University to use this test. For example, cost, access to data, flexible independent scheduling and the ability to measure TOEFL equivalence. The SLEP test is a product of the Educational Testing Service (ETS). ETS also provides the TOEFL test and consequently provides a relatively reliable approximation scale for translating SLEP scores into equivalent TOEFL scores. Since the SLEP test is significantly easier than the TOEFL test and the students are primarily at an elementary level, there is a strong argument that the SLEP is more appropriate than the TOEFL.

Findings
Utilising the outlined methodology, the findings provide quantitative survey and SLEP test proficiency data that allow for discussion on the hypothesis as to whether English Internet usage is an indicator of progress.

Survey data
In this section we outline the findings of the survey. The first two sections profile the subjects in terms of social background and technological capability. The third section focuses on findings relevant to the hypothesis.

Background information
The cohort is of first year university students from one university in the Kanto area of Japan. The cohort was to a large degree an opportunity sample. The sample size was 121 subjects. In terms of gender the survey was heavily weighted with male respondents. Approximately 80% of the cohort were male. This gender in-balance is a reflection of the gender in-balance within two of the departments surveyed, Economics & Political Science, and Local Community Policy Studies. Unfortunately this heavy male weighting means that the study does lack gender balance. This fact will have implications for generalising the results to the broader society. Almost 91% of subjects surveyed were aged 18-19 years old. A further 97.5% indicated that Japanese was their first language. This cohort consisted of students from nine compulsory proficiency levelled classes in three departments. The nine classes include three A, B and C/D level classes. There is little distinction between the C and D level classes in terms of proficiency.

General internet and CMC usage
There was an overwhelmingly positive response to Japanese domain Internet usage. Over 99% of respondents use the Japanese domain Internet. In fact only 1 respondent from a cohort of 121 stated that they do not use the Japanese domain Internet. Furthermore, the frequency of usage was also highlighted as very strong. Over 90% of subjects use the Japanese domain Internet at least once a week. Almost 53% of the subjects use the Japanese domain Internet everyday. Finally, it would seem the students have a need for and access to the Japanese domain Internet as the duration of usage was also highlighted as a strong factor. The Internet sessions of 55% of subjects were at least one hour in duration. Given the usage findings, it was not surprising to find that 88% of those surveyed agreed with the statement that the Japanese Internet was an important part of their life. Indeed, 50% strongly agreed with this statement. Moreover, when questioned on how they use the Internet, 90% of the respondents used the Japanese Internet for their university studies, such as research for assignments and presentations. In a similar result, 90% stated they used the Internet for entertainment, such as listening to music or watching movies. Further still, 90% suggested they used the Japanese Internet for daily life activities such as checking the weather, sports results, and train schedules. Yet, in contrast, only 34% stated that they used the Internet to receive the daily Japanese news and 55% used the Internet in work related ways, such as job-hunting.
In terms of Japanese CMC, over 87% use e-mail, the most popular CMC tool available. A further 61% participate in social networking, such as Mixi or Facebook and 42% use a messenger service such as Skype or Yahoo messenger.

Progress data
In this section, we outline the findings of the pre-test and post-test SLEP data with reference to progress. Firstly, the scores have been translated from the SLEP scaled scores to the TOEFL paper test scores based on a chart provided by Educational Testing Service (ETS, 2008). ETS is the creator and provider of both SLEP and TOEFL tests. This was undertaken to attempt to make the data more understandable and meaningful to the majority of potential readers.
In order to understand the data, when transferring the SLEP scores to TOEFL scores we find that the lowest score possible in TOEFL terms is 200 points. During the pre-test phase, the lowest score was 200 points using the TOEFL scale. During the post-test phase, the lowest score was 205 points on the TOEFL scale. In terms of the highest score during the pretest phase, the highest score was 380 points, whilst this increased to a high of 450 points during the post-test phase. Indeed, 12 students bested the pre-score high with scores in the range of 390-450 points.
Furthermore, according to ETS (2008), score progress in the range of -20 to +20 points are not a reliable measure of improvement or decline. Consequently, 67 subjects made no recognisable decline or gain with their scores falling in the ±20 point range.
That said, there was significant progress to report. Overall, 47 subjects improved significantly, between 25 and 170 points. Furthermore, seven subjects fell significantly by 25 to 50 points.

Non-Internet causes for progress effect
It is important to look at alternative causes to English domain Internet usage for the effect on progress. As will be shown, the entertainment variable appeared to be masking the results and the unmasked subset of students became the focus of this section of the data findings.
If we remove the English domain subjects who only use the English domain Internet for entertainment, our group of 17 users falls to 12 subjects. The reasons for doing this will be explained in the discussion. This group became a focus of attention and that attention Improved No change Declined is manifested in the final sections of these findings. The following sections will focus on these 12 key subjects.
The ranking data provides an interesting view of progress. Initially the 12 key subjects were ranked on the pre-test between 11th and 85th position. On the post-test this had changed to between 3rd and 61st position. This included five of the top ten ranked students. The pre-test data showed that the subjects were spread widely across the cohort ranking range, after the post-test the subjects became mostly clustered into two distinct groups at the top and middle of the range.

Figure 2. Ranking of 12 key subjects
The subjects were selected from three departments, Euro-American Cultural Studies, Economics and Political Science, and Local Community Policy Studies. The 12 students that appeared to be making a strong and varied use of the English domain Internet were mostly in Euro-American Cultural Studies, with nine of the 12 subjects were members of this department. Two subjects were members of the Economics and Political Science department and one subject was a member of the Local Community Policy Studies department.

Teachers
The study included nine classes taught by five teachers. The 12 key subjects were found in all five teachers' classes. The majority of the key subjects were clustered in the teachers of A level and Euro-American Cultural Studies classes. Indeed, five key subjects were taught by one teacher in the A class of the Euro-American Cultural Studies Department.

Discussion
The following is a commentary of general and English domain Internet and CMC usage among the respondents.

General Internet and CMC usage
At least 99% of the surveyed subjects use the Internet and 90% of the cohort were 18-19 year old university freshmen. In Japan university age students are more likely to use the Internet via a smart-phone than a personal computer. This finding would seem to support the claims of Chinnery (2006) and the findings of Thornton and Houser (2005), regarding 100% mobile telephone usage among Japanese college age students. Whilst the Internet is a more generic technology and mobile smart-phones are a specific form of CMC technology capable of utilising the Internet, it is clear that 99% of the cohort is using CMC technology.
Interestingly, Internet World Stats (2011) suggests 78% of Japanese speakers are using the Internet. In this study, 99% of young Japanese people are using the Internet. The findings suggest that the Internet has become a focal point of many of the subjects' lives. For example, whilst over 90% of the cohort use the Internet every week, a very significant number, 53%, use it every day. This has implications for future applications of the Internet. Yet the extent to which the Internet plays a role in the subjects' lives can be examined further. The respondents are not only frequently using the Internet, but 55% are staying online for over an hour per sitting. Whilst not wishing to underestimate the life of a Japanese teenager attending university, it would seem reasonable to suggest a large amount of their time is spent studying and socialising. Consequently, the Internet appears to be playing a central role in their lives since over 90% of respondents use the Internet for both study and entertainment purposes.. This assertion is supported by 88% of the respondents, who agreed that the Internet was important to their life. The Japanese respondents usage of specific CMC tools was actually quite surprising. The fact 61% of respondents are using or have used social networking tools and 42% use or have used a messenger service was a surprise. Further research is required to understand the extent to which these tools are being utilised. However, once again, it does point to a CMC based future.

English Internet and CMC usage
Firstly, 14% of the respondents claim to use the English domain of the Internet. This was far higher than anticipated. The fact that 10% of the entire cohort claim to use English every week and 5% use it every day was equally surprising. We appear to have a large number of Japanese youth utilising English materials on a regular basis through their own volition, which was very unexpected. 14% of the respondents state that the English Internet is important to their life. This appears to be supported by the frequency, as described, and duration of their usage. 6% of respondents use the English Internet for an hour or more and a further 6% use it for at least 30 minutes duration. In terms of CMC tool usage in English the results were rather interesting. E-mail, at 5%, was the least used of the three CMC areas questioned in contrast to being the most used in Japanese Internet usage (87%). E-mail is a first-generation tool and it would seem the second-generation tools are more appealing for English usage with 6.5% using social networking, and 6% using messenger services. However the number of subjects was low, so possibly the findings could be a little misleading. If we allow for a margin of error of 2% there would be no significant difference. However, when findings are analysed from the perspective of user populations we find that messenger service use amongst the cohort's English Internet users (42%) and Japanese Internet users (42%) are comparable. However, e-mail amongst the cohort's Japanese Internet users was 87% compared to only 35% of English domain users. Given that the social networking comparison appears to fall between the e-mail and messenger services, English (46%) and Japanese (61%), this may suggest that the more technically savvy the user, the more likely they are to use English. However, social networking was the most popular CMC tool in English, so popularity of technologies could also drive English usage. If these points are valid they could have implications for a future increase in English needs of Japanese users or they may mean that only a relatively specialised niche require English. Finally, 11% of the cohort are using the Internet for their university studies and a staggering 28% claim to be using the English Internet for entertainment. The latter result is both a puzzle and, as will be shown in the next section, the key to a greater understanding of student English Internet usage and its relationship to progress.

Analysis of the survey and progress data
Once the survey data is cross-referenced with the progress data an interesting picture reveals itself. In regards to the hypothesis, two strands emerge. Firstly, there is evidence to support the hypothesis. Secondly, one area of Internet use appears to mask the results and raises questions about how we should use the Internet for study purposes. However, it is equally important to look at variables that may be boosting our perception of the English Internet's impact on EFL progress. In this sense the question becomes does English L2 Internet lead to EFL progress or does EFL progress lead to further L2 Internet?

Is English Internet usage an indicator of progress?
17 subjects answered question 19 in the affirmative, that they do use the English Internet. These 17 subjects had a mean average progress gain of approximately 35 points. This compares very favourably with the cohort mean of 22 points. Thus the group of subjects who answered a simple yes to using the English Internet did 59% better than the full cohort. This is further supported by student rankings in the pre and post tests. Whilst in the pre-test the best placed student was 11th, in the post-test, five of the 17 subjects placed in the top ten. This result, in and of itself would appear to strongly support the case for the implied correlation stated in the hypothesis. However, it is possible to refine and improve the results further. We need to confirm that the student progress is caused by English Internet usage.
The following sections will help to clarify, if this is indeed so.

Entertainment masks and filter
A further 23 subjects answered no to question 19 (see appendix), stating they do not use the English Internet, yet from questions 23-31 which are yes/no questions regarding English Internet usage, they answered yes to one or more. Evidently this would seem contradictory. Firstly, a further 23 subjects answering yes to some English Internet usage meant that 40 from 121 subjects (33%) claimed to be using the English Internet. This was a surprising result. Whilst not wishing to caste aspersions upon the subjects' integrity, it would be reasonable to consider what the subjects meant by English Internet usage. This led to the uncovering of the masking effect of question 25, using the English Internet for entertainment. Nine of the 23 subjects answered no they do not use the English Internet and then reconsidered, answering yes to one further question, question 25, that they actually do use the English domain of the Internet for entertainment.
Question 25 appeared to be significant and was causing respondents to contradict themselves. It appeared plausible that question 25 may be masking the results. There appeared to be a need to control for this variable. For example, for the 17 subjects who answered yes to question 19, stating they use the English Internet. If from the 17 subjects, the five subjects who then continued on to answer yes to only one more yes/no question, question 25, entertainment, are filtered out, we find that the mean average score gain of the remaining 12 subjects rises to 46 points. That is more than 109% better than the cohort average gain.

Figure 5. Average proficiency progress gains
Furthermore, the 12 subjects include four of the cohort's top ten gainers and six in the top 20. Eight of these subjects made gains of 40 points or more. Significantly none of the 12 subjects had a negative progress score. In fact, in the pre-test the lowest ranked student among the key 12 subjects was 85th, however, in the post-test the lowest ranked member of the key 12 subjects was 61st. Upon reflection, the entertainment variable was always likely to have a masking effect as the entertainment, primarily music and videos, would have been in general usage off-line as well as online. English music and movies are easily available in Japan. If all the subjects are doing is transferring typical, off-line, life experiences of society to their online life, then little gain could be expected. In this sense entertainment is background noise.
The idea of entertainment as background noise appears to be supported by the nine subjects who stated they do not use the English Internet and yet went on to state that they do use the English Internet for entertainment. The 18-point mean average gain for the nine subjects is comparable to the 22-point gain of the 121 subject cohort.

Other masks and filters
The data suggested a strong correlation between the key 12 subjects and the Euro-American Cultural Studies department. 75% of the key subjects compared to 36.36% of the full cohort were members of the Euro-American Cultural Studies department. This may support Ryan's (2009) assertion that the Japanese primarily study English because they like it. However, the nine Euro-American Cultural Studies members averaged approximately a 48-point gain. This compared favourably to the departmental gain within the cohort of 28.52 TOEFL points. Therefore, there are most likely variables other than departmental affiliation in play.

Figure 7. Average Euro-American cultural studies progress gains
Seven of the 12 subjects were members of A level classes. This may support the view that progress and higher proficiency led to English L2 Internet usage rather than the hypothesised reverse. However, the seven subset A class members progressed an average of 44 points compared to their A class peers average gain of approximately 21 points. Consequently, it seems unlikely that A class membership alone can explain the gains.

Figure 8. Average A Level progress gains
The 12 key subjects were spread across all five of the teachers whose classes were surveyed. This would lead to the suspicion that individual teachers played little role in the results. However, five subjects were clustered in the A level Euro-American Cultural Studies teacher's class. This most likely points to the importance of the class level and department rather than the teacher. The Euro-American Cultural Studies departments A class cohort subjects gained 33 points, significantly better than the departmental gain. In contrast, the five-subject subset of A level Euro-American Cultural Studies students, who use the English domain of the Internet, was a strong gain of 46 points. Once again, even with two variables in combination it does not seem to fully explain the gains. Increasingly, it would appear likely that the results suggest that there is a variable involved outside the classroom. conclusion This study appears to have confirmed a relatively high usage of the English domain of the Internet. This has allowed us to discuss the likely claims of our hypothesis. We cannot categorically state that L2 Internet usage leads to progress and it certainly seems true that high proficiency students do appear to be more likely to use the L2 Internet as one would expect. However, it is also true that the results suggest that the students who claim to make use of the English domain of the Internet appear to make significantly more progress than their peers compared by either full cohort, department, class level or actual class. In many ways, the results have exceeded expectations. To find such a high percentage of English domain users among the cohort was of real interest. The fact that our subset of key subjects made at least 50% more progress than their peers compared at each of the class, departmental and full cohort levels, suggests something is probably happening. This argument is supported further still, as when we focused on a combination of variables, the dominant departmental affiliation and higher-class level, we still found that the key subjects averaged almost 40% more progress than the selected grouping.
To further develop and corroborate the findings of this research, it would be desirable to undertake further research. Recommendations would include a more gender balanced and far larger cohort possibly sampled from a wider group of universities. We may also look more closely at the question of if English L2 Internet leads to EFL progress or does EFL progress lead to further L2 Internet? The research tends to portray a mixed view. On the one hand, the A-level students were more likely to use the Internet than the lower level students, although not exclusively. On the other hand, regardless of level, the English domain Internet students saw substantially greater progress gains than all comparable peer groupings examined. This would suggest that English Internet domain usage can lead to further L2 progress. However, in this research the emphasis has been on classroom variables that may affect the hypothesis. Consequently, there is a strong need to look at variables outside of the classroom that may have influenced the otherwise compelling results. This would need to be the main thrust of further research. Further still, taking this the logical step forward, we may want to examine closely what exactly the students are doing on the Internet. The aim of this research was to gain a greater understanding of the effects of the Internet on English proficiency and progress. The findings and discussion seem to have shed light on these effects. The students are actually using the English domain. The progress data backs up the survey data. The fact that 14% of students are using the English domain of the Internet and 10% are using it for more than entertainment was a very interesting result. There are clear implications for the future and the effect CMC appears to be having on student English usage patterns. However, CMC is likely to move rapidly into new arenas and drive the social experiences of the coming decades in ways that are hard to predict. Consequently, there is now and most likely always will be an evolving real need for further research.

appendix
The Full Survey Results