Gesture-Based Cartesian Control with Real-Time Feedback for Object Positioning and Screw Insertion

M A HAFIZ

Keywords: gesture control, human-robot interaction, Cartesian control, EMG, IMU, teleoperation, robot manipulation

TL;DR: Hybrid EMG+IMU gesture interface enables intuitive Cartesian control with real-time feedback for precise robot manipulation.

Abstract: Most gesture-based robot control interfaces map to joint commands or lack precise task-space feedback. We present a gesture-based Cartesian control framework that integrates EMG and IMU signals with an SVM-based classifier (15 s per gesture training). Users control end-effector translation (X, Y, Z) and rotation (roll, pitch, yaw) through a hierarchical interface that combines IMU-based menu navigation with gesture-based axis selection and positive/negative motion commands in task space. A real-time Cartesian error feedback mechanism provides positional differences with respect to the target, enabling iterative refinement during task execution. The system is evaluated on a Franka Emika FR3 robot with three users performing object positioning and screw insertion tasks. The SVM classifier achieves high recognition accuracy (97--98\%) across users. Positioning errors decrease from approximately 200 mm to 3--12 mm within five trials, while screw insertion achieves 4--18 mm accuracy with rotational error below 0.15$^\circ$. These results suggest that gesture-based Cartesian control with real-time feedback can support intuitive control, rapid user adaptation, and preliminary performance in contact-rich manipulation tasks.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 25