Recent advances in digital technology, both in research prototypes and commercial products, have introduced a plethora of multitouch horizontal display surfaces. Perhaps because most of these devices are flat, initially most of the multitouch interfaces were two-dimensional (2d) in nature. However, on traditional tables, people frequently make use of the third dimension;​ they pick up, turn over, stack, build, and otherwise manipulate objects on physical tables. Furthermore, they frequently use the visual cues made available by the third dimension, such as viewing the different sides of an object or scene, or hiding something underneath another object. Successful interaction with three-dimensional (3d) objects on tabletops involves both manipulation and visual feedback. Thus, I simultaneously explore the research questions of both viewing and interacting with 3d virtual artifacts on digital tables.

The use of 3d virtual objects on a tabletop display introduces many research questions. For example, most applications that support 3d graphics do so by assuming a single viewpoint directly in front of the display. This assumption is no longer valid when using a large horizontal surface that affords many people working at different sides viewing the 3d virtual scene. It is an open question to what extent this discrepancy in viewing angle is problematic, and, when necessary, how it can be mitigated. Furthermore, the horizontal table imposes a physical barrier to the 3d virtual world, meaning that “touch” input will be within the 2d plane. Another open question is how this 2d information can be used to control 3d virtual objects “below” the table’s surface and whether interacting through this surface can enable the kinds of 3d abilities common to physical artifacts.

To address the research questions about the discrepancy in viewing angle between the different people around a table and the viewpoint used to render a scene, I empirically study this perceptual phenomenon. Results show that, in a tabletop display setting, viewing projected 3d virtual objects from multiple viewpoints is indeed problematic and becomes more problematic as the discrepancy in viewing angle increases. In this dissertation, I describe how to apply these results to 3d applications, either through an understanding of the compromises implied by each design or by using mitigating techniques to reduce the problem.

In this dissertation, I also build on previous work that explores manipulation of a virtual 3d object by introducing several techniques which use the 2d touch input provided by multiple fingers or contact points. Results of a comparative user study showed that both performance and preference increased as participants were provided with more touches to control the virtual objects. While this study only explored 2d movement and 3d rotation (the techniques did not allow lifting of virtual objects), the insight gained was used to create sticky fingers and opposable thumbs, which extend the three-touch technique to allow lifting (the sixth degree of freedom). By combining the power of this full control over any 3d virtual object with physically-based reactions of other virtual objects and interface components, sticky tools provides a framework for 3d tabletop interfaces that eliminates the need for specialized gestures or an abstract menu system.

This framework, together with insights gained from the exploration of viewpoint discrepancy, were applied to the practical application of enabling sandtray therapy, a form of art therapy for children, on a digital table. This application was cooperatively designed with therapists who use sandtray therapy in their regular practice. This application serves as a demonstration of how to apply the concepts in this dissertation to the design of 3d interaction on a tabletop display.