Model-Based Switched Approximate Dynamic Programming for Functional Electrical Stimulation Cycling

Wanjiku A. Makumi, Max L. Greene, Kimberly J. Stubbs, Warren E. Dixon

Abstract: This paper applies a reinforcement learning-based approximately optimal controller to a motorized functional electrical stimulation-induced cycling system to track a desired cadence. Sufficient torque to achieve the cycling objective is achieved by switching between the quadriceps muscle and electric motor. Uniformly ultimately bounded (UUB) convergence of the actual cadence to a neighborhood of the desired cadence and of the approximate control policy to a neighborhood of the optimal control policy are proven for both motor control and muscle control via a Lyapunov-based stability analysis provided developed dwell-time conditions that determine when to switch between the motor or the muscle are satisfied. Lyapunov-based techniques are also used to derive a minimum dwell-time condition to prove UUB stability of the overall switched system.