An augmented multi-touch system using hand and finger identification

Peter Kung, Dominik P. Käser, Craig A. Schroeder, Tony DeRose, Donald Greenberg, Kenrick Kin

Abstract: With the advent of devices such as smart phones and tablet computers, multi-touch applications are rapidly becoming commonplace. However, existing multi-touch sensors are not able to report which finger, or which hand, is responsible for each of the touches. To overcome this deficiency we introduce a multi-touch system that is capable of identifying the finger and hand corresponding to each touch. The system consists of a commercially available capacitive multi-touch display augmented with an infrared depth camera mounted above the surface of the display. We performed a user study to measure the accuracy of the system and found that our algorithm was correct on 92.7% of the trials.