Design and user evaluation of a mobile application to teach Chinese characters

The Open University qian.kan@open.ac.uk This paper presents both the pedagogical design principles and the challenges faced during the development of a mobile app, Chinese Characters First Steps, to support recognising, learning and practising writing Chinese characters at beginners’ level, including the difficult balance between pedagogy and technical affordances. We also present the results of a research study which collected quantitative data from over 130 users of the app as well as qualitative data from interviews with four users. The results offer a profile of language learning app users and very positive attitudes towards learning with apps. They also reveal users’ reasons for learning Chinese, patterns of learning, expectations of the app, and evaluation of the different features for their own personal learning. Finally, the paper concludes that the chosen design principles for the app were appropriate for its purpose.


introduction: the app revolution
When the first generation iPhone was introduced in 2007, few could have predicted the impact it would have on education.As Godwin-Jones (2011) points out, the iPhone was a game-changer not only in terms of its functionalities, but because other competitors followed and created similar devices.Since then, and with the addition of the iPad -introduced in 2010 -and similar devices, there has been a proliferation in the popularity and ownership of smartphones and tablets that can carry out a large variety of educational activities within a single device.Although the devices are clearly necessary for this to happen, the other pillar of this revolution in mobile learning is the availability of the apps developed to be used with them.
As soon as these apps were released, educators began to see the potential for teaching using such devices, including ubiquitous learning opportunities, ease of use, collaboration, content generation, and productivity enhancement (Murphy, 2011).But does this potential become reality?And what are the implications for language learning in general and for more complex languages such as Chinese?

literature review
This section will present the challenges of learning Chinese, current theory on the use of mobile apps for language learning, and the evidence found so far in research studies about their efficacy for teaching and learning Chinese.

Challenges of learning Chinese
It is well established in the research of teaching Chinese as a foreign language (cfl) that, due to its unique features, it presents the following key challenges to language teachers and learners: i) tones; ii) character recognition and writing; iii) lack of obvious correspondence between the character script and the sound (Hu, 2010;Liang & van Heuven, 2007;Wang & Kirkpatrick, 2012).Tones and characters are two of the biggest hurdles at beginners' level (Liang & van Heuven, 2007;Jongman et al, 2006;Hu, 2010;Xiao, 2010).Not only are there too many similar sounding words in Chinese (Lü, C, 2010), but also many words sound exactly the same (with the same syllables and same tones) but mean different things and have different character representations.Learning characters is crucial in meaning identification.However, characters are perceived as difficult because of their sheer number (Hu, 2010) and complexity of strokes (on average a character consists of about 12 strokes).Many beginners, face-to-face or learning at distance, feel that they are learning two systems at the same time -pinyin (phonetic transcription of the characters) and characters.In this process, they need to do four mappings (see Table 1 with an example for the word for 'west'): Although there has been extensive research looking into character learning strategies (Ke, 1998;Lu, 2009;Yin, 2003) as well as issues concerning tone production and recognition (Liang & van Heuven, 2007;Jongman et al, 2006;Hu, 2010), not many researches have looked into the effective use of new technologies to support and engage learners in doing the four mappings as illustrated in Table 1.

Apps for language learning
2.2.1 Potential and challenges.Many authors have highlighted the potential of smartphone and tablet devices as well as mobile apps for language learning, based on the theoretical principles and evidence from the field of Mobile-Assisted Language Learning (mall) (Burston, 2014;Godwin-Jones, 2011;Golonka et al, 2012;Kim & Kwon, 2012;Kim, 2013;Lafford, 2011;Lys, 2013;Shinagawa, 2012;Sweeney & Moore, 2012;Yang & Xie, 2013).This potential derives from features such as larger screen size, responsive touch screen, enhanced text-entry, high-quality audio and video playback, picture, video and audio recording and editing, voice recognition, enlarged storage, faster connectivity, gps, accelerometers and Bluetooth connections (Godwin-Jones, 2011), all within one device.
Criticisms of the design of language learning apps include too many translations, lack of sound or pictures, poor navigation and user-interface design, and little use of the unique properties of smartphones such as the motion sensor, location detector, and, in particular, connectivity with other users (Godwin-Jones, 2011;Sweeney & Moore, 2012;Burston, 2014).Burston (2014) found that language learning activities on mobile apps have mostly replicated what had been done before with other technologies and limited to basic flashcards, multiple choice, blank filling, drag and drop and vocabulary and grammar drills and games.He concluded that "pedagogically, nothing new has been done with smartphones that has not already been done with earlier mobile devices" (Burston, 2014, p. 108).In their review of language learning mobile apps, Kim & Kwon (2012) highlighted that most apps focus on cognitive processes (recognition, recall and comprehension) and receptive language skills rather than socio-cognitive activities, with little collaborative learning; they also found the activities not as well developed as they could be and mostly teacher-directed, with focus largely on vocabulary.

Evidence of app use for
Chinese language learning.Although some educators and students are quick to adopt new technologies into their teaching and learning, it is necessary to evaluate the real potential of the tools.
As Golonka et al (2014) correctly point out: "in spite of an abundance of publications available on the topic of technology use in fl learning and teaching, evidence of efficacy is limited" (Golonka et al, 2014, p. 70).Studies on the use of mobile apps for Chinese language learning are scarce.Shinagawa (2012) described some of the benefits that mobile apps can bring to learning non-western characters.These include the iPhone's own Chinese handwriting recognition function, dictionaries, and quizzes.Among the attempts to find evidence of Chinese learning with apps, Wong et al (2010) carried out a research study in Singapore, providing their students with smartphones to photoblog Chinese idioms and subsequently base collaborative activities around them.They encountered varied levels of participation (partly due to parental negativity towards the use of smartphones by school 西 west children) as well as technical problems.Despite this, their students reported more engagement with Chinese idioms -although post-study performance was not tested.Yang and Xie (2013) also researched learning Chinese idioms, using iPads with heritage learners studying at a university in the usa.The participants had very positive impressions of the learning experience.A post-test after the activity showed that almost all students could recall the idioms practised, however, after two weeks, the recall had fallen to 40%.As discussed in section 2.1, character recognition and writing present a challenge for learners of Chinese as a foreign language.Technology can facilitate their learning or provide methods to help overcome the need to handwrite them (Lai & Gu, 2011), but very few studies have looked into whether and how apps can support character recognition and writing.Huo (2013) found that stroke order is still relevant when learning Chinese with the use of technology, and claimed that stroke order helped memorise characters.Based on the research she carried out with learners of Chinese at a us university, Chung (2013) claimed that practising Chinese characters with an iPad had led to the students learning basic stroke order, familiarising themselves with structural components of the characters, and gaining knowledge of the history and culture behind the words.Her students liked the fun, game-like features of using the iPad and this encouraged them to continue with the activities.Some of her students commented on the ease and convenience of using a mobile device for learning, for example whilst commuting, allowing them to make the most of their time outside the classroom.Chung concluded that using the iPad facilitated the integration of learning Chinese into the students' daily lives.

Design principles and development of the app
Beginners' Chinese was introduced to the curriculum at The Open University (ou), a ukbased distance learning university, in November 2009.As distance learners, our students face additional challenges to those mentioned in 2.1, such as lack of physical and visual presence of tutor and fellow students (Hurd, 2005); lack of immediate personal feedback, and limited face-to-face speaking and interaction opportunities (Kan & McCormick, 2014;Stickler & Shi, 2013).Due to these challenges, the beginners' Chinese course team searched for innovative tools to support their students in meeting the challenges of learning tones and characters.However, in 2011, there was limited number of apps designed for Chinese character learning.Amongst those in the market at the time, most of them had less than 50 randomly selected characters.The team was not able to identify an app that was designed to learn characters in a progressive and personalised manner.To address this and other shortcomings of the apps at the time, such as over-emphasising the fun element (showing the rain drops with the character for rain) whilst neglecting integrating different aspects of learning characters: pronouncing, listening, writing and recognising characters, the team worked with the app developers at our university to create a new app 'Chinese Characters First Steps'.Although the app content is of use to students taking the ou course, its use is not compulsory.This is because the university does not assume that students will have access to (or own) a smartphone or tablet that can run the app.This section presents the principles behind the app design and the stages of production that the app went through.

Design principles
Sweeney & Moore (2012) stated that the critical success factors in the design of apps for language learning are pedagogy and technology.The principles behind our app design included: Ȼ Bite-size learning and mobility It was agreed that small learning bites would work best due to small screen size and devices often being used on the move, as they can also maximise the number of learning situations in which the resource can be used (Kovalchick & Dawson, 2003).In addition, the mobile learning experience has been described as "highly fragmented" (Trifonova & Ronchetti, 2003) since its early days, and best design should take this into account by providing activities that can be completed in a short amount of time to make use of 'dead time'.Hence, each listening and learning activity in the app consists of only ten matching questions.

Ȼ Progressive learning
There are altogether 20 lessons with 400+ characters in the app, and each lesson consists of an average 20 characters.All the characters are covered in the ou beginners' Chinese course, which are the most commonly used characters at beginners' level.The 20 lessons build upon each other in the same chronological order as in our course materials so the characters learnt in previous lessons are used to build new words in later lessons.
Ȼ Integrating writing, listening, reading, and vocabulary building Each lesson consists of four sections: i) Writing -about 20 characters that learners can learn to draw with their fingers using the correct stroke order indicated on the screen, they can also hear how to pronounce them; ii) Listening test -learners hear ten randomly chosen words or phrases, then select the correct answer from the choices displayed; iii) Reading test -learners see ten randomly chosen words or phrases in characters, then select the correct answer from the choices displayed; and iv) word search -most lessons have two word search puzzles.Each puzzle consists of 16 characters that can form about eight words/phrases.(See Figure 1, below, for screen shots.)Ȼ Gaming feature -an element of fun Each listening and reading test is timed at 60 seconds as a default, which can be reset to 40 or 20 seconds.The learner can see the clock count down, making it feel like playing a game.For word search puzzles, learners use their fingers to draw a line horizontally or vertically over the characters that form the correct word/phrase.If it matches the word/ phrase in the list, a pink line appears, otherwise, the line disappears.

Ȼ Personalised learning
Learners can set the level that suits them (easy, normal or hard).The default is Easy with stroke order indicator and 60 seconds per test.If users choose a harder level, the time available shortens and the stroke order indicator disappears.As users complete each character, data is kept within the device memory to allow the users to see how many times they have practised it.

Development of the Chinese Characters First Steps app
Although not much research had been done at the time on the potential of using mobile devices in language learning, the project team felt that the affordances of smartphones and tablets with touch screen were ideally suited to a language like Chinese.For the first time,

Research questions
The research study reported here aimed to a) understand the learner experience of using the Chinese Characters First Steps app and b) evaluate the principles used for the design of the app.
The researchers had gathered some first impressions from users of the app by looking at the reviews and ratings left on the uk and us App Stores.On the us App Store, 12 ratings were received, with an average of 3.5 stars (five 5-star, two 4-star, two 3-star, one 2-star and two 1-star ratings).A further 12 ratings were posted on the uk store.There, the app received an average of 4 stars (seven 5-star, two 4-star and three 1-star ratings).The positive reviews praised the appropriate level for beginners, variety of skills practised and the user interface.Low scores and negative reviews were received from users who had encountered technical problems with sound, tracing or payment (these reviews had been left prior to the bug fixes released with versions 2.2 and 2.3 of the app).Whilst this was of interest, a much more detailed account of the user experience was needed.
Most studies into the use of apps for language learners have been carried out with the researchers' own students as participants (e.g.Wong et al,2010;Yang and Xie,2013).Teachers know their students and base their teaching strategy on that knowledge.When the design is aimed at massive online consumption, that knowledge is non-existent.Investment and time is being spent on developing language learning apps, but developers, businesses and educators cannot design well without some knowledge of who users are and what they want.This research project aims to understand who the end-users of language learning apps are, using data both from students registered on the ou Beginners' Chinese course and from other users external to the institution who had downloaded the app.
The research questions were: 1.What is the profile of the app user? 2. What is their motivation for using the app? 3. How do they use the app? 4. What are their expectations and impressions of using the app? 5. Do they think that using the app has improved their Chinese language learning?

Methods
To obtain feedback from app users, a prompt was added to the app as a pop-up, which appeared after users utilised the app more than five times.In addition, an email was sent to a uk-based Chinese language teaching mailing list asking teachers to encourage their students to take the survey.Furthermore, a message with the survey link was posted on the course forum asking students registered on the ou beginners' Chinese course to take the survey, and finally a tweet with the survey link was sent from the course Twitter feed encouraging followers to take the survey.
When users chose to take the survey, a link took them to the site where it was hosted.The survey (using SurveyMonkey) consisted of 25 questions, of which 24 were multiple-choice and one was an open comment question (see Appendix A for full survey).
The survey ran between June 2012 and March 2014.A total of 137 responses were collected.All respondents finished the survey.The only question which did not receive full responses was the open question, which was answered by 69 participants.The data gathered were analysed using SPSS 21 to provide descriptive statistics.A second data analysis was carried out using a question which asked participants whether they were registered on a course in Chinese as a variable to ascertain whether there were any noticeable differences between formal and informal learners.Many of the responses in terms of use and impressions of quality were very similar.Where appropriate, the results section will highlight the responses where there were marked differences between the respondents.
To further explore how learners utilised the app, four users of the app were interviewed in June 2013.The users were registered on the ou Beginners' Chinese course and had indicated that they had been using the app in response to another research study on mobile language learning (one of the authors of this paper participated in the study).The interviews were carried out using a semi-structured schedule (see Appendix B for sample questions) via Skype or telephone, with each interview lasting between 30 and 50 minutes.The interviews were recorded and transcribed.Using a thematic analysis of the interview transcriptions, the data were examined to identify salient features about the pedagogical design principles of the app and whether they facilitated the learning experience.

Participants
The profile of the survey respondents is as follows: 79 (58.5%) were male and 56 (41.5%) female.With regards to age, the largest group of participants were aged between 30 and 39 (Figure 2).The largest proportion of respondents live in the uk (34.3%), followed by other European countries (21.6%),North America (usa, Canada) (15.7%),China (11.2%),Australia and New Zealand (8.2%), other Asian countries (3.0%), Central and South America (2.2%), and Africa (0.7%).A further 3.0% indicated they live elsewhere.Only 17.2% of the respondents were registered on the ou beginners' Chinese course, the remaining 82.8% were not.

Results and discussion
In this section the results from the data collected will be presented and discussed in relation to the research questions.

What is the profile of the app user?
A high proportion of the survey respondents were male (58.5%).This is in line with uk undergraduate population studying Chinese and consistent with the student profile of our university Chinese course (Kan & McCormick, 2014) as well as conventional uk universities (hesa, 2011), where more men study Chinese in comparison with European languages.With regards to age, most participants were older than the traditional university-age students who had taken part in previously reported research, however, the age distribution percentages were very similar to those of the students taking the course at our institution, who are also older adult learners.
Whereas most students taking the course live in the uk (as expected from a uk institution), we found marked differences between those app users who were registered on a course and those who were not.As expected, most respondents who are registered on a formal course live in the uk because the ou is a uk-based university and therefore the app is more widely promoted in the uk than throughout the rest of the world.Informal learners were much more highly represented in the us, Canada and China.This is not surprising as it would be unlikely for respondents who live in those countries to register with a uk university, whereas beginner Chinese learners who live elsewhere, China in particular, would be likely to want to support their learning with additional resources such as a mobile app.The majority of respondents (89.7%) described their current level of Chinese language as beginner (35.0% as "absolute beginner" and 54.7% as "beginner") which is the audience the app is targeted at.A further 8.8% selected "intermediate" and two people selected a higher level, one (0.7%) "upper intermediate" and another one "advanced".

What is their motivation for using the app?
The survey asked participants why they were learning Chinese.The question allowed more than one answer and 69.3% responded that they were learning the language because of personal interest, as is common with many language learners (Coleman, 2009).In addition, 46.7% did it because it was an intellectual challenge, and a further 33.6%indicated that it was relevant to their profession.Such a high proportion was expected given that China is a nation that has recently experienced enormous economic growth and now attracts a higher volume of business than previously.Heritage learners (9.5%) stated that they had family ties with China, and 8.8% were learning Chinese because it was relevant to their current studies.These are usually highly-motivated learners who actively seek additional resources such as mobile apps to support their learning.
All respondents indicated that they used the app for learning rather than teaching.A total of 25.9% of respondents were formal learners registered on a Chinese course (either from the ou that the app supports or elsewhere), so the app was mostly used to learn Chinese informally: either as the main source of learning (as was the case for nearly 17% of the participants) or as support to other informal sources of learning (likely to be books or other online resources such as websites, podcasts and / or other apps).This confirms Khaddage & Lattemann's finding (2013) that a vast majority of students who owned smartphones use apps as additional learning resources.Among the users who were not registered on a Chinese course, the proportion of respondents who claimed that they were trying to learn informally was much higher (80.5%) than among those who were registered on a course (37.5%).Similarly, the difference between those who considered the app their main source of learning was much higher (24.4%) among those not registered than the respondents who were registered on a course (3.7%).
As all the four interview participants were registered students of the ou Beginners' Chinese course, they used the app as a revision tool to revise, reinforce and test the characters learned on the course.One participant (Interviewee A) used the app mostly for recognising characters, saying it helped "to reinforce […] to revise the characters".Another participant (Interviewee C) used the app in conjunction with the course materials.He said that he tried "to learn all the words and do everything in line with the study" (see 6.3 for how they use the app).

How do they use the app?
Of the ios devices that the app runs on, 70.8% used an iPhone, 32.1% an iPad, 11.7% an iPod touch and 5.1% selected "other".Some used more than one device.
The survey asked the participants the number of lessons in the app they had completed at the time they took the survey.Just under three quarters of the respondents had used the app for more than one lesson, meaning they must have purchased the upgraded version with 20 lessons (including the 62% who knew they had upgraded and some of those who were unsure about whether they had upgraded or not).This is a high percentage, but must be taken with caution as it makes sense that users who had upgraded would make more use of the app and perhaps be more willing to take part in a survey about it.
With regards to how long at a time users normally spend utilising the app, most respondents (72.8%) used the app for 15 minutes or less.Some 2.2% selected "less than 5 minutes", 9.6% "about 5 minutes", 36.0%"between 6 and 10 minutes", 25.0% "around 15 minutes", 17.6% "between 16 and 30 minutes" and 9.6% "over 30 minutes".This is consistent with the design principles for our app.
The interview data shows that participants responded well to the bite-size approach to delivering content and the possibility of using the app during 'dead time'.One interviewee (Interviewee A) commented that the app was "useful for filling in time", whereas another (Interviewee B) mentioned she made sure she took her iPod with her during commute time so that she could use the app to learn characters.A further participant (Interviewee C) found that the advantage of having the app available was "the speed that you can get access" to fill in 10 or 15 minutes of spare time, which otherwise may be wasted.
The interview data also shows that on the whole participants worked progressively lesson by lesson but at the same time took advantage of the app design to work on any section or character they wished to focus at a time.The design feature of being flexible and personalised within the framework of progressive learning was welcomed by all four participants.As Interviewee C commented that he "might just pick one [character], one of the ones I've done" sometimes, but overall he went lesson by lesson because "I want to keep doing it properly".This is further supported by Interviewee B: "I think I just see what I fancy on the day and what I have been struggling with.If there is something that I know we have learnt but I can't remember I will go back to that."This further supports the findings in the literature on flexible design for distance language learning (Bates, 2005;Garrido, 2005).Garrido (2005) points out that in designing distance course 'there had to be a high degree of flexibility built into the course to meet wide-ranging expectations ' (2005, 180); whilst at the same time 'distance learners need a teaching and learning framework that engenders a high level of motivation to help them stay on track ' (2005, 184).

What are their expectations and impressions of using the app?
To elicit data on user impressions of the app, the survey asked participants about their expectations and how these were met.There was a wide range of expectations about the app.Based on the features of the app, four categories were offered as choices 1-4.In addition, a further two choices (5 and 6) were added involving an unrealistic expectation to become fluent in Chinese and having no expectations respectively.Respondents could tick all that applied.The results show that most respondents hoped to learn to write some characters, followed by learning to recognise some characters (see Table 2).It is interesting that a much lower percentage of respondents expected to learn to identify sounds (choice 2) or say words (choice 4).This could be linked to the app name and description, both of which refer to characters rather than sounds or pronunciation, or there may be an assumption that using apps for learning Chinese is more apt for reading and writing rather than aural skills.It was interesting to see that a -very small -number of respondents indicated that they expected to become fluent in Chinese by using an app with 20 lessons.On the one hand, this is clearly an unrealistic expectation -certainly in the case of this particular app -on the other hand, however, it suggests that some people may believe that studying a language with an app on their personal device can lead to fluency, which is something that would not have occurred five years ago.
The next question asked participants whether the app had met their expectations (see Table 3).These results vary for those respondents who were registered on a Chinese course and those who were not.These differences between the two groups suggest that those not registered on a course had more realistic initial expectations of the app, as 53.8% found the app as they expected it to be.This seems logical for users who, as informal learners, are likely to utilise several apps and perhaps have more realistic expectations of what an app can do, which is consistent with the claim that learners often engage in meaningful language learning activities by using several apps (Lafford, 2011).Of the three respondents who selected "worse than expected", one had selected "I would be fluent in Chinese" as his/ her expectation.It is pleasing that 97.7% of respondents indicated that the app had either met or exceeded their expectations, as it is evidence that the app is fit for purpose.Again, this supports findings from previous studies in the literature that users have positive attitudes towards using language learning apps (Kim, 2013;Khaddage & Lattemann, 2013).
In the open comment question, participants were asked what else they would like from the app.Some 69 respondents answered the question.The most popular request was for more characters/content, with 25 responses.This was predictable.Among these, six specified they wanted content that was more challenging, including full sentences, a further six specified they wanted more content on tones, and three wanted more pronunciation practice.A total of four respondents requested more free content.Some 16 respondents suggested new features: the most popular request (from six different respondents) was the ability to test themselves on writing characters without having a model answer in front of them.Other features suggested included the ability to interact with other learners and an interface in a language other than English (two respondents each), a dictionary (two), being able to personalise the interface, and writing tests.Finally, four respondents would like detailed feedback on the quizzes.It is interesting that some of the suggestions for additional features, such as the ability to interact with others and detailed feedback, are consistent with features that had been highlighted in the literature as desirable from language learning apps.Some of the above suggestions have been addressed in the latest version of the app.
All four participants in the interview said they liked the app.Interviewee C said "I really like that app […] It's the favourite one that I've found".Depending on their own individual learning needs, they liked different functions of the app: some liked the stroke-by-stroke writing of characters most whilst others like the reading test/listening test or word search most.They also made some good suggestions for improvement such as adding audio button to each character in the Writing section so that the user can listen to the character before they practise the writing, which was implemented into the latest version of the app.

Do they think that using the app has improved their learning?
Participants were asked to rate the overall quality of the app (see Table 4).With over 90% of participants rating the app as either 'good' or 'very good', it was clear that they had very positive feelings about learning with it.Not all features were rated equally: whereas the quality of the app for learning to recognise Chinese characters and read Chinese words received very high ratings (over 86% rated it 'good' or 'very good' for recognising characters and nearly 88% gave similar ratings for reading Chinese words -formed by several characters), the quality of learning to write Chinese characters received a generally positive but lower satisfaction rate (72.4% rated it 'good' or 'very good'), possibly due to i) the frustration in drawing some complex characters on a small screen (as suggested by the interview data); and ii) the inherent difficulty of learning to write Chinese characters as opposed to just recognising them.
With respect to learning to write Chinese characters using the app, and whether this skill was transferable outside the app, participants were asked whether they thought that practising writing characters on screen helped them to write on paper as well.In total, 48.9% selected "a lot", 31.9%"a little", and 8.9% "not much".Nobody selected "not at all" and 10.4% had not tried to write the characters on paper after practising with the app.This suggests that participants believe that repeated writing of characters, whether using one's fingers on a screen or with pens on paper, facilitates the production of characters.
Finally, the participants were asked whether using the app had improved their knowledge of Chinese.No gradation of improvement was offered as options other than "yes", which 96.3% selected, or "no", selected by 3.7%.The perceived benefits of learning characters with the app are further supported by the interview data, where all the four participants who had used the app found the app a useful learning and revision tool.Interviewee A reported that the app enabled him "to try and learn more of the characters more thoroughly".For Interviewee B, she found the practice of stroke order useful "because I never remember which order the strokes go in.And also I like the reading practice because characters are difficult to remember."Interviewee D mentioned that using the app is "just a different way of reviewing characters and sounds".As stated earlier, over 80% of participants felt that practising writing characters on the app helped them, to some extent, to write characters on paper as well.The above findings are consistent with the literature that learners reported that learning with mobile devices helped the recognition and writing of Chinese characters (Huo, 2013;Chung 2013).

Conclusion
The results from this study have found that the app we created has been used in the way it was designed for and led to very positive impressions of learning with it.The app helps character recognition and writing as it combines the stroke-by-stroke writing of characters, sound, pinyin and English translation on one screen to reinforce the learning; it allows users to use the app when they have a small amount of time or when they are on the move; and by giving them the freedom as well as the structure to suit their own learning needs, they are more likely to engage with the learning.
The study has also provided one of the first profiles of users of apps for language learning and their usage habits.It is very positive that such a varied pool of users utilise the app, as it confirms the appeal of apps for language learning regardless of gender or age.This research is necessary in the field of language learning apps to understand the way learners engage with them.The feedback and data collected from users for this study will be useful to inform the design of further developments.
Sufficient evidence in the data suggests that the design principles used in the development of the app have been useful to learners.During the development of the app, one of the key challenges was to bridge the gap between the language teachers' limited technical knowledge and the app developers' limited of knowledge of both pedagogy and the Chinese language.It was crucial to communicate clearly and frequently on the pedagogy and learning design.The language experts needed to be ready to embrace new ideas but at the same time strike the right balance between technical affordances and pedagogy, for example by not spending an unnecessary amount of time trying to achieve some technical novelty at the expense of the learning outcomes.Both sides had to be prepared to change the specification of the app as development went on because some ideas did not work out as expected, whilst other ideas came up during the development process.For example, the word search activity was not in the initial design of the app, but was borne during the process as we tried to combine the principle of creating an element of fun with progressive learning, which turned out to be a feature very much welcomed by users.Finally, whereas the academics were accustomed to taking longer periods of time to prepare resources, app developers worked at a fast pace.Because of this, it became important to respond quickly to queries and suggestions and adapt to their availability to work on the app design.These are valuable lessons to future designers of language learning apps, regardless of whether they are commercial or educational content providers.

Limitations
The study asked participants to indicate what their impressions of learning with the app were.Whilst this is helpful to understand how users feel about using the app as a learning too, it does not provide any evidence of actual learning.In principle, the data from participants who were taking the course could have been correlated with their performance on the course assessment.A number of obstacles impeded this.Firstly, some were using other sources of learning in addition to the app, so the issue of whether the learning was due to app use or not would have been unclear.Secondly, the confidentiality of users might be compromised as it would be necessary to correlate the survey responses with their attainment.Thirdly, being distance learners, there would be no way of ascertaining that the person who took the survey was the same person taking the assessment.Finally, it was important to keep the survey confidential to avoid any perception of bias or participants thinking that taking part or "good" or "bad" responses may influence their grades.In addition, assessment scores would not have provided evidence of learning, as it is assumed that a large proportion of their learning is due to completing the course and not just based on the app.
It was also impossible to gather analytics from the app itself on number of attempts or levels achieved and link it to the data from the survey.This is something that can be easily integrated into app design now but was not available at the time of the app development.

Further research
Work continues to improve the app: we have recently produced an Android version and updated the ios version (released on the iTunes Store in August 2014) to meet new os system requirements and to include usage analytics.The data gathered from the analytics, user feedback and suggestions collected for this study will inform the design of further developments.We will continue to undertake research based on later versions of the app.
We believe that further research is necessary in the field of apps for language learning, and apps for Chinese language learning in particular.More information is required to understand the way learners engage with apps for language learning, in particular when a proportion of these use apps as their only source of learning resources.Such research would benefit from screen recordings and eye-tracking technology as well as observation of behaviours and/or learning analytics of app activity (such as number of attempts or levels achieved, as discussed above).Although the participants in this survey reported very

Figure 1 .
Figure 1.Screen shots for Writing, Listening and Reading sections and the Word search section

Figure 2 :
Figure 2: Age of participants

Table 1 :
An example illustrating the four mappings learners need to do to match pinyin spelling, pronunciation, character script and translation.

Table 3 :
Responses to the question "Has the app met your expectations?"

Table 4 :
Responses to the question: "Please rate the overall quality of the Chinese Characters First Steps app"

quality Writing characters Reading characters Recognising words
Are you currently registered or have you ever taken the L197 Beginners' Chinese course from The Open University?Yes No 4. Are you currently registered on a Chinese course (not at the ou)?Are you trying to learn Chinese "informally" (i.e. by yourself without signing up to a course)?Yes No 6.On which of the following devices do you use the Chinese Characters First Steps App? (tick all that apply) Why haven't you upgraded your app to unlock all the content?Too expensive Not worth it I want to know more about the content before I pay for it I never pay for apps I wasn't sure how much I'd have to pay if I tried to upgrade I tried but had technical problems doing so I was not aware that I could upgrade the App 11.How many of the lessons in the App have you attempted?What expectations of the app did you have when you downloaded it?(tick all that apply) I would learn to write some Chinese characters I would learn to identify some Chinese sounds I would learn to recognise some Chinese characters I would learn to say a few Chinese words I would be fluent in Chinese I had no expectations 15.Has the app met your expectations?Please rate the quality of the Chinese Characters First Steps app for learning to write Please rate the quality of the Chinese Characters First Steps app for learning to recognise