Abstract. This paper describes the result of user evaluation of a location-aware travel guide prototype. The prototype consists of information that is commonly needed for travellers or tourists when they arrive at a particular destination. The prototype was implemented into a Nokia Symbian Series 60 phone with Location API (JSR-179) support. The evaluation was carried out in Arau and Kangar, two urban areas in the state of Perlis. Participants reported high satisfaction with the content provided, even though few usability problems were encountered while interacting with the prototype. The findings from the evaluation indicate that mobile phone users were fascinated with location-based information that could assist them along their journey.
Keywords: mobile guide, user evaluation, Location Based Service, tourism
Nowadays, mobile devices has become common in our daily lives, due to the advancement of technology that made it reasonably priced and thus ubiquitous. Mobile phones are not only restricted to making calls or receiving multimedia messages, but also provide other information-based services and infrastructures [1,2,3]. Part of infrastructure introduced in current mobile devices is Location-based Service (LBS) which enables mobile application to integrate spatial coordinates with a wide range of services . LBS make it possible for developers to produce application that is aware of its current location and present users with a list of choice that allows them to interact with the environment. For example, if a user is at a bus station, an LBS-enabled mobile application would allow user to select between viewing the bus schedule, ticket prices and the option to buy them. The user would benefit from the association of his current location and the action that he can perform within the surroundings. Additionally, LBS has the advantage of being available on the technology-neutral platform and thus, the resulting application could be adapted on conforming devices . The number of LBS applications has been projected to rise over time as more LBS-capable devices are shipped into the market .
One of the most popular utilization of LBS is tourism, where it is used to enhance user experience while travelling by providing users with forms of interaction or services based on the users' location . This development has lead to a number of studies that looked into LBS within the domain of tourism, for example the CRUMPET personalised tour guide at Heidelber and Helsinki  which focuses on integrating user personalization and data communication framework for mobile devices. GUIDE  electronic tour guide at Lancaster on the other hand focus on developing and evaluating a location aware mobile tourist guide by using browser metaphor as a tool for user interaction. GUIDE prototype however uses a specialized unit based on Intel Pentium machine in contrast of the more generally available mobile phone.
While studies done on CRUMPET did not include user evaluation and satisfaction studies , the result of its findings regarding data personalization has been used as a basis of this research. AccesSights on the other hand concentrates on building a mobile tourist guides for sighted and blind people . Both AccessSights and GUIDE uses specialized device which are probably not accessible through standard mobile phone available today.
This paper describes the knowledge gained from user evaluation of a mobile tourism guide project which was carried out in the urban area of Arau and Kangar, Perlis, Malaysia. The prototype was first designed using paper prototyping technique [6,11] to discover usability problems in its early design. This technique has helped the researchers to minimize corrections that need to be done on the user interface when it has been coded.
Even though it was reported that there were no significant difference in the usability problems that were discovered from both lab and field test [12,13], we chose to do the evaluation in the field. The reasons were 1) the prototype has the capability to detect the location of the user and display information of the adjacent areas only, as such it would be more appropriate to do the test in the field rather than in a fixed laboratory setting, 2) we wanted to perform a hallway usability test, in which we could simply ask members of the public randomly to test the prototype rather than recruiting participants in advance and 3) we preferred the users to use it in a real context such as while sitting at a caf or while standing by a parking space. For that reasons, the findings section will discuss the average execution time for every task, observation of user behavior while using the prototype in a public space and their acceptance towards such application.
The prototype was field tested in the campus of a university and a college as well as near shop lots in the town area. In order to get participants, we walked around the campus and asked a few colleagues to test the prototype. We also went to the caf and library of a college nearby the university and shop lots around the town area to find volunteers who were interested to test the prototype. 13 participants ranging from the age of 18 to 39 years old took part in a three-day trial. They came from technical and non-technical background - students, computer technicians, IT officer, library staffs and lecturers. They were six female and six male who have been using mobile phones for more than six years, and one female student who has used a mobile phone for only three years. All participants were familiar with the Nokia mobile phones and they have never participated in any usability test before. There were five tasks to be accomplished by every user. The instruction for every task revolves around finding and viewing information from the prototype while being stationary at a public space. This was due to the fact that we did not have the special equipment to support video recording while the user is walking, as mentioned by [12, 13]. The tasks were created to reflect the common needs of a traveller or visitor who has not planned the trip ahead or who has no access to local information. They were viewing information of available accommodation, restaurants, facilities and services, places of interests and shopping hotspots.
One of the researchers acted as the facilitator who conducted the session - introduced the prototype to the user and read the tasks to be completed. One researcher became the observer who observed the user while he or she was executing the tasks and the other used a stopwatch to measure the execution time to complete a particular task. Every test session was video recorded to assist the researchers in analyzing user behavior later. The users were given instruction to inform the facilitator if they could not accomplish the task so that the facilitator could skip to the next task. They were also encouraged to explore the prototype without feeling anxious from the presence of the observer and the timekeeper. After testing the prototype, every user was informally interviewed to gather his or her feedback, comments and suggestions.
Results and Discussion
This section will discuss the average execution time for every task and the observation of user behaviour and their acceptance towards the application.
Task execution time
User Behavior and Acceptance
From the video recording analysis, it was learned that most participants were confused when they first looked at the screen interface. This was because the main menu of the prototype used a fish-eye style  instead of list style user-interface, thus participants tried to figure out on how to navigate around. However, after pressing the navigation key (up,down & left,right), they understood quickly that they were supposed to press the left or right key to browse through the main menu. The younger participants (aged 18-24) navigated smoothly and seemed very comfortable while executing the tasks. In contrast, those aged above 30 (30-39) took quite a while to become accustomed to the screen interface despite having more experience in using mobile phones. The observation indicated that there are lot to be improved in the user interface of the prototype so that it can provide an enjoyable experience for its users, regardless of age and background. As a result of the informal interview with all participants, it was discovered that all participants were enthusiast with the idea of having a mobile travel guide that disseminates useful information of adjacent areas relative to their current location. As with the user interface, 76.63% of the participants agree that the fisheye user interface implemented in the main menu helped them to navigate around the application as it provide visual cues through the use of icons and labels. 38.46 % of the participants agree that the main menu needs to be improved by enlarging the icons to fill up mobile phone screen and to provide more visual cues such as the left and right arrow icon beside the fisheye menu to assist users in using it.
From the results, we can conclude that users from all age-group welcomed the LBS application can provide useful information and services to their current location. The call function of the mobile tourism information needs to be improved by placing a telephone icon as to make it easier for users to place a call after they have found their desired information. Older users seem to take much more time to figures how the application works when they first use it when compared to the younger users. The older participant prefers on-screen tips to use the application compared to the younger participants that are comfortable with the user-interface that uses icons as a visual cue. It can be concluded that the overall user interface of this application have to be designed to appeal users that are most likely a frequent traveller in order to make the application much easier for them to use.
The prototype can be further improved in terms of its contents and screen interface. The interface should provide more visual cues to guide users to use the applications. The application should display a map from their current position to their place of interests to guide them to the location. The user interface of the prototype needs to be improved in ways that it users could perform their tasks in the shortest amount of time by taking into account on the target users age group for the application.
- E. Kaasinen. "User needs for location-aware mobile services" Personal and Ubiquitous Computing Journal, vol. 7, pp. 70-79, May 2003.
- W.S.Yang, H.C. Cheng and J.B. Dia. "A location-aware recommender system for mobile shopping environments," Expert Systems with Applications Journal, vol. 34 Issue 1, pp. 437-445, January 2008
- B. Patten, I. A. Snchez and B. Tangney. "Designing collaborative, constructionist and contextual applications for handheld devices," Computers & Education Journal, vol. 46 Issue 3, pp. 294-308, April 2006.
- J.Schiller and A.Voisard. Location-based services. San Francisco: Morgan-Kauffman, 2004, ch.1, pp. 10-21
- M. Hinz, Z. Fiala. "Context Modeling for Device- and Location-Aware Mobile Web Applications", 3rd International Conference on Pervasive Computing (Pervasive 2005) PERMID 2005, Mnchen, May 2005
- M.H. Ismail, A. Osman. "Location Aware Travel Guide: Refining Design Via Paper Prototype", Proc. of Conference on Scientific & Social Research 08'09, Melaka,Malaysia, March 13-15 2009.
- M. Hazas, J. Scott., J. Krumm."Location-Aware Computing Comes to Age". IEEE Computer, vol. 37 Issue 2, pp.95-97, February 2004.
- S. Poslad, H. Laamanen, R. Malaka, A. Nick, P. Buckle, and A. Zipf. CRUMPET: Creation of user-friendly mobile services personalised for tourism. In Proceeding of 3G 2001 - Second international conference on 3G mobile communication technologies, pages 28-32, London, UK, 2001.
- K. Cheverst, N. Davies, K. Mitchell, A. Friday, and C. Efstratiou. "Developing a Context-aware Electronic Tourist Guide: Some Issues and Experiences". In Proceedings of the 2000 Conference on Human Factors in Computing Systems (CHI-00), 2000, pp. 17-24
- P. Klante, J Kroche and S Boll. AccesSights - A Multimodal Location-Aware Mobile Tourist Information System". Computers Helping People with Special Needs. New York: Springer, 2004, pp. 627-634
- C. Snyder. Paper Prototyping: The Fast and Easy Way to Design and Refine User Interfaces. New York: Morgan-Kauffman, 2003.
- A. Kaikkonen, T.Kallio, A.Keklinen, A. Kankainen and M. Cankar. "Usability Testing of Mobile Applications: A Comparison between Laboratory and Field Testing," Journal of Usability Studies, vol. 1, pp. 4-16, November 2005.
- J. Kjeldskov, M.B. Skov, B.S. Als and R.T. Hegh. "Is it Worth the Hassle? Exploring the Added Value of Evaluating Usability of Context-Aware Mobile Systems in the Field", in Proc. of MobileHCI04, Glasgow, Scotland, Sept 2004.
- B.B. Bederson, "Fisheye menus". In Proceedings of the 13th annual ACM symposium on User interface software and technology (UIST-00), 2000, pp.217-225