Smartphones have become a constant companion at home and at work, and more and more elderly people want to use them for everyday tasks like messaging or managing appointments and events, too. However, the user interface is usually not designed for their needs, and smartphones in general are often perceived as too complicated by the elderly. The question which arises is how such an interface should be designed. One way of tackling this issue is to try to integrate technology into everyday surroundings and let it fade into the background. Hence, we developed a concept of an alternative user interface for smartphones which is based on familiar techniques – pen and paper. Instead of using a conventional pen however, our approach is based on a digital pen that is connected to the smartphone and enables the transfer, analysis and digital processing of written text.

Our proposed concept introduces three components: the digital pen, a notebook and a smartphone. While the user is writing and interacting with the notebook, the camera-equipped digital pen continuously transmits its positions on the paper, which is equipped with a fine dot pattern for this purpose, to the connected smartphone. The notebook consists of different pages which contain pre-defined input fields that enclose different entities (e.g. example messages, contacts, calendar days or certain commands). Different tasks, like for example the creation of calendar entries, sending an SMS or email, calling a contact or creating a medication reminder, require a different set of input entities. Based on the selected entities, the smartphone will automatically choose the most reasonable task. While interacting with the notebook, the smartphone is responsible for guiding the user through the tasks and performing resulting actions (e.g. sending an SMS or setting up a reminder). The display of the smartphone, which represents different states of the system with different colors, serves as the main feedback component.

The following example shows how the sending of an SMS message works:

message-sending

The user selects a contact in the notebook by tapping on it with the pen. The smartphone receives the selected position of the pen and is thus able to map it to the corresponding person. The user then begins to write a text in the provided input field. The strokes are instantly transferred to the smartphone and processed via Optical Character Recognition (OCR). At this stage, the smartphone assumes that the user wants to send a message. When the user taps on the “send” button, the smartphone composes and sends the SMS to the recipient that has been selected before. Such tasks can be aborted at any time by tapping the “cancel” button.

Publications

2012 T. Birn – PaperPhone: Papiergestützte Eingabemethode für Smartphones zur Anwendung durch ältere Menschen – Master’s thesis, University of Applied Sciences Upper Austria, Austria, 2012. [pdf]

Researchers

Thomas Birn*, Christoph Schaffer
Department of Mobile Computing, University of Applied Sciences Upper Austria, Austria
thomas.birn [at] fh-hagenberg.at

* … student in the Master’s degree programme Mobile Computing