A Vision-Based Low-Cost Power Wheelchair Assistive Driving System for Smartphones

Zhiwei Wang, Kevin Liu, Jeffrey Wang, Jingye Xu, Jingjing Chen, Yufang Jin, Rocky Slavin

Abstract: Power wheelchairs (PWC) are essential for people with mobility impairment, and many research studies have been reported to ease their operations. However, the existing approaches either rely on extra hardware components or demand complex software that incurs high costs. In this work, we propose a low-cost, computer-vision-based assistive driving system that runs on a smartphone with the objective of safely driving a PWC in a hands-free manner with reduced attention in an indoor environment to relieve the arduous operations of disabled users and reduce their stress. The system adopts a modified and pre-trained ResNet-50 model running on a smartphone to derive the driving instructions using the images captured in real-time with its built-in camera. The smartphone interacts with a control interface to send the driving instructions to the PWC. A prototype of the proposed driving assistive system is implemented on a Pixel-6 Android phone and evaluated on a mobile robot as the proof-of-concept design. The experiments show that the smartphone can process input at up to 3.4 images per second to generate driving instructions in time to safely navigate the mobile robot at reasonable speeds in the testing environment with minimal intervention from the user.