Paper Reading #20 - A Multimodal Labeling Interface for Wearable Computing

Comments:
//comment 1
//comment 2

Reference:
A Multimodal Labeling Interface for Wearable Computing
Shanqing Li, Yunde Jia
IUI '10

Summary:
With the advent of wearable computing, there are new possibilities and obstacles to consider.  Among these obstacles is the ability to adapt to new environments without using a keyboard and mouse for interaction with the wearable device.  This paper seeks to use visual and audio modalities as a substitute for mouse and keyboard interaction.

The system utilizes the following componants: a head mounted display, a Bluetooth microphone, a stereo camera, a wearable computer, and a virtual touchpad.  A user uses the virtual touchpad to trace an object, which creates a model (much like a cardboard cutout).  The model is created when the user encircles an object, using the same motions they would on a chalkboard.  A speech recognition system with a 98% recognition rate is used to label the new model.

In evaluating this system, the researchers found that the mean error in tracking the movements was .046m, with a standard error or .043m.  In addition, they found that their system proved to be much faster than using a keyboard and mouse, with their system performing even better as larger objects were labeled.

Discussion:
There are interesting possibilities with this system.  Although it is currently a rather bulky and unwieldly package, in time it can be used in a variety of ways.  In the paper, they had images of labeled landmarks.  The physical text on the landmarks was in a foreign language, but the labels were in english.  While especially invaluable for tourists, this could be utilized to track out of the way places in a crowded city.  In addition, people could use it to make virutal breadcrumbs to find their way back after visiting an unfamiliar area.  I look forward to the possibilities that this technology will offer.