Sensing how people interact with their mobile phones under real-world conditions can be an expensive and time-consuming task, especially if data from a large number of users and over a long period of time is needed. However, in order to improve the utility and usability of mobile applications, field studies are often considered to be more suitable than laboratory evaluations. Thus, we developed an innovative approach towards the automatic and transparent sensing of mobile phone usage, which is based on the background observation and analysis of user interactions with a mobile application under real-world conditions.

We have implemented a software framework for this purpose, which can easily be added to an Android application. It records how users interact with the application and transmits the acquired data to a server. The Java-based aspect-oriented framework AspectJ has been used for the implementation, which has the advantage that developers do not have to change application code in order to use interaction logging in their mobile apps. We evaluated our framework with regard to its performance and memory consumption on the one hand, and by adding it to an application in the marketplace on the other hand. The results from about 300 users over one week showed that the framework runs stable and with very low resource demands.

The following example shows how interaction logging works for a simple application:

Interaction Logging

(1) A user starts an application and a login screen appears. A first interaction event is logged as soon as a new screen is displayed in portrait mode. The user touches the first input text field and enters his name. The user continues with a touch on the field for the password. (2) Subsequently, the user touches the log in button, resulting in the pop-up of a progress dialog. (3) After authentication has been completed, the user is redirected to a new screen. The following log output shows the interaction data which is received by the server for this example:

Logger started (Tue Aug 20 17:07:58 MESZ 2013)
Framework Kernel V2.0.0 running on REL, V4.2.2 (17)
– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –
17:07:59 CONTEXT [OrientationService]: Set orientation to: Portrait
17:07:59 INTERACT [ActivityService]: Starting screen: LoginScreen
17:08:08 INTERACT [WidgetService]: View clicked [LoginScreen]: EditText “at.test:id/et_username”
17:08:23 INTERACT [WidgetService]: View clicked [LoginScreen]: EditText “at.test:id/et_password”
17:08:23 INTERACT [WidgetService]: View clicked [LoginScreen]: Button “at.test:id/btn_login”
17:08:33 INTERACT [ActivityService]: Starting dialog: [AuthenticationDialog]
17:08:38 INTERACT [ActivityService]: Hiding dialog: [AuthenticationDialog]
17:08:39 INTERACT [ActivityService]: Stopping screen: LoginScreen
17:08:40 INTERACT [ActivityService]: Starting screen: RestrictedAreaScreen

The software framework consists of a central administration unit (Kernel) that is capable of managing so called services. Basically, each service fullfills a distinct logging task (e.g. touch positions, user navigation or context). As these modules only depend on the Kernel, all services are exchangeable and can be adopted for various use cases. Additionally, the framework becomes easily extensible without the need for modifications of the Kernel. Thus, it can also be used to record interactions for custom hardware like mobile operator terminals for example. The collected interaction events are synchronized based on NTP time and sent to a server, where developers or designers can checkout in realtime what users are currently doing in an application.

Publications

2012 F. Lettner, C. Holzmann – Automated and Unsupervised User Interaction Logging as Basis for Usability Evaluation of Mobile Applications – Proceedings of the 10th International Conference on Advances in Mobile Computing and Multimedia (MoMM2012), Bali, Indonesia, 2012. [acm] [pdf]
2012 F. Lettner, C. Holzmann – Sensing Mobile Phone Interaction in the Field – Proceedings of the 10th IEEE International Conference on Pervasive Computing and Communications (PerCom 2012) Workshops, Lugano, Switzerland, 2012. [ieee] [pdf]
2011 F. Lettner, C. Holzmann – Usability Evaluation Framework: Automated Interface Analysis for Android Applications – Proceedings of the 13th International Conference on Computer Aided Systems Theory (EUROCAST 2011), Las Palmas de Gran Canaria, Spain, 2011. [springer]

Partners

Various partners from industry in the FFG fundeded projects AIR and AUToMAte.

Researchers

Florian Lettner, Clemens Holzmann, Christoph Purrer*, Patrick Hutflesz
Department of Mobile Computing, University of Applied Sciences Upper Austria, Austria
florian.lettner [at] fh-hagenberg.at

* … student in the Master’s degree programme Mobile Computing