Portalware: Exploring Free-Hand AR Drawing with a Dual-Display Smartphone-Wearable Paradigm

Open Webpage

Jing Qian, Tongyu Zhou, Meredith Young-Ng, Jiaju Ma, Angel Cheung, Xiangyu Li, Ian Gonsher, Jeff Huang

Published: 2021, Last Modified: 04 Nov 2025Conference on Designing Interactive Systems 2021EveryoneRevisionsBibTeXCC BY-SA 4.0
Abstract: Free-hand interaction enables users to directly create artistic augmented reality content using a smartphone, but lacks natural spatial depth information due to the small 2D display’s limited visual feedback. Through an autobiographical design process, three authors explored free-hand drawing over a total of 14 weeks. During this process, they expanded the design space from a single-display smartphone format to a dual-display smartphone-wearable format (Portalware). This new configuration extends the virtual content from a smartphone to a wearable display and enables multi-display free-hand interactions. The authors documented experiences where 1) the display extends the smartphone’s canvas perceptually, allowing the authors to work beyond the smartphone screen view; 2) the additional perspective mitigates the difficulties of depth perception and improves the usability of direct free-hand manipulation; 3) the wearable use cases depend on the nature of the drawing, such as: replicating physical objects, “in-situ” mixed reality pieces, and multi-planar drawings.