Paper Reading #7 - Manual Deskterity: An Exploration of Simultaneous Pen + Touch Direct Input

Comments:
http://jaiachi.blogspot.com/
http://chi-garza.blogspot.com/
Reference:
Manual Deskterity: An Exploration of Simultaneous Pen + Touch Direct Input
Ken Hinckley, Jerry Rodenhouse, Koji Yatani, Michel Pahud, Andy Wilson, Hrvoje Benko, Nicole Coddington, Bill Buxton
CHI 2010

Summary:
With the proliferation of touch-screen technology, there are many products available that allow either pen input or touch input.  However, there are few products available that incooperate both.  Manual Deskterity is a scrapbooking application that lets a user write with a pen, manupulate with touch, and explore other options with a combination of the two.

To determine the design of this project, eight test subjects were given a notebook and some creative materials and told to design a short film.  The behaviors of the subjects were observed to determine how a typical user naturally utilizes pen and paper in combination.  With few exceptions, the actions observed were uniform across the test subjects.

The foundation of Manual Deskterity's design lies in these observations.  Objects can be created, deleted, and manipulated through touch, while the pen is utilized for writing.  Specific modes are avoided by giving each object a radial context menu.  Touch and pen controls can be combined to perform actions such as stapling items together, cutting and tearing items, copying items, and using an item as a straightedge or ruler.

Since the design of Manual Deskterity is based on real-world behaviors, the gestures used to interact with the system seemed obvious and intuitive to users.  However, the designers believe that designing a more general purpose system would be difficult without adding layers of complexity to the design.

Discussion:
I am excited by the possibilities this research opens up.  One of my personal gripes with touch-screen technology is that precision movements are difficult to achieve with just one or two fingers.  Furthermore, fingers can obscure what someone wants to interact with, leading to unneccisary frustration.  On the other hand, a pen based device seems unwieldly when broader motions are required.  In combining the two we get the best of both worlds.  One thing I would like to see more research on is how left handed people would interact with the system.  Lefties typically hold a writing instrument in a radically different manner than their counterparts, so I suspect that the sensors would be thrown off by this variation.  Barring that, I look forward to a commercial use of this software with anticipation.