Current interactive surfaces do not support user identifi cation. Hence, personalized applications that consider user-specifi c access control are not possible. Diverse approaches for identifying and distinguishing users have been investigated in previous research. Token-based approaches – e.g., which utilize the user’s mobile phone – are especially promising, as they also allow for consideration of the user’s personal digital context (e.g., stored messages, contacts, or media data). However, existing interaction techniques are limited regarding their ability to enable users to manipulate multiple items at the same time (e.g, select or copy).

We developed MobiZone, a novel approach which enables users to interact with large numbers of items on an interactive surface while enabling personalized access by using the mobile phone as a token. MobiZone provides each user with a spatial zone that can be positioned, resized and associated with any action according to the user’s needs; items enclosed by the zone can be manipulated simultaneously. In order to use MobiZone, the user has to connect the phone to the surface first. The mobile phone serves as a token that allows for identi fication to the surface, thus enabling personalized access to specifi c items. At the same time, the phone serves as a versatile tool to perform diff erent kinds of actions that can be customized with the phone. Users can change the zone’s X/Y position and size, as well as the actions (e.g., transferring items from the phone to the surface) that are applied to the items within the zone. Multiple items such as photos or contact cards, which are located within the spatial zone on the surface, can thus be manipulated simultaneously.

We developed three different interaction techniques which allow users to control the position and size of a spatial zone: FlashLight&Control, Place&Control and Remote&Control. We used Microsoft PixelSense as the surface device (Surface 2.0 SDK). For the communication between mobile phone (Windows Phone 7.5 SDK) and surface, a TCP socket connection was established. In order to track the phone’s position above the surface for the FlashLight&Control interaction technique, we used a Kinect sensor (Kinect SDK 1.0) which forwarded sensed data to the surface via USB. Additionally, the orientation of the phone, which was determined via its bult-in accelerometer, compass and gyroscope sensors using the combined motion API, was sent to the surface. For the Place&Control technique, byte tags were attached to the back of the phone in order to detect its position on the surface.

Images

FlashLight&Control allows to set the size as well as the position of the spatial zone by translating or rotating the mobile phone.

FlashLight&Control allows to set the size as well as the position of the spatial zone by translating or rotating the mobile phone above the interactive surface.

Place&Control allows to control the spatial zone by placing the phone on the interactive surface. Its size can be adapted with a pinch-gesture, and the position of the zone is bound to the mobile phone and follows the device.

Place&Control allows to control the spatial zone by placing the phone on the interactive surface. The position of the zone is bound to the mobile phone and follows the device.

Remote&Control initially places the zone at a random position. Users can resize the zone and drag-and-drop it to a new position.

Remote&Control initially places the zone at a random position on the interactive surface. Users can resize the zone with a pinch gesture and drag-and-drop it to a new position.

Publications

2013 M. Rader, C. Holzmann, E. Rukzio, J. Seifert – MobiZone: Personalized Interaction with Multiple Items on Interactive Surfaces – Proceedings of the 12th International Conference on Mobile and Ubiquitous Multimedia (MUM 2013), Luleå, Sweden, 2013. [acm] [pdf]

Partners

Research Group Human-Computer-Interaction, Ulm University, Germany

Researchers

Markus Rader*, Clemens Holzmann
Department of Mobile Computing, University of Applied Sciences Upper Austria, Austria
clemens.holzmann [at] fh-hagenberg.at

Enrico Rukzio, Julian Seifert
Research Group Human-Computer-Interaction, Ulm University, Germany
enrico.rukzio [at] uni-ulm.de

* … student in the Master’s degree programme Mobile Computing