Making Real-World Interfaces Accessible Through Crowdsourcing, Computer Vision, and Fabrication

Anhong Guo, Jeffrey P. Bigham

Abstract: The world is full of physical interfaces that are inaccessible to blind people, from microwaves and information kiosks to thermostats and checkout terminals. Blind people cannot independently use such devices without at least first learning their layout, and usually only after labeling them with sighted assistance. To address this problem, we introduce VizLens---a robust and interactive screen reader for interfaces in the real world. VizLens users take a picture of an interface they would like to use, it is interpreted quickly and robustly by multiple crowd workers in parallel, and then computer vision is able to give interactive feedback and guidance to users to help them use the interface in real time. Built on top of VizLens, we developed automatically generating tactile overlays to physical interfaces to provide blind people with a permanent static solution. We introduce Facade---a crowdsourced fabrication pipeline to help blind people independently make physical interfaces accessible by adding a 3D printed augmentation of tactile buttons overlaying the original panel.