Towards Autonomous Berry Harvesting using Visual Servoing of Soft Continuum Arm

Shivani Kamtikar, Samhita Marri, Benjamin Thomas Walt, Naveen Kumar Uppalapati, Girish Krishnan, Girish Chowdhary

Keywords: Soft Continuum Arm, Visual Servoing, Berry Reaching

TL;DR: This paper demonstrates a deep-learning based visual servoing technique for a soft continuum arm as a step towards autonomous berry harvesting.

Abstract: Autonomous berry harvesting is a challenging problem, especially with hard-to-reach targets inside the plant. Using soft continuum arms is a step towards achieving this task without causing excessive damage to the plant. Visual servoing is a popular control strategy that relies on visual feedback to close the control loop in controlling a soft arm. However, robust visual servoing is challenging as it requires reliable feature extraction from the image, accurate control models and sensors to perceive the shape of the arm, both of which can be hard to implement in a soft robot. This work circumvents these challenges by presenting a deep neural network-based method to perform smooth and robust 3D positioning tasks on a soft arm by visual servoing using a camera mounted at the distal end of the arm. A convolutional neural network is trained to predict the actuations required to achieve the desired pose in a structured environment. An integrated approach for estimating the actuations from the image is proposed. In addition, a proportional control law is implemented to reduce the error between the desired and current image as seen by the camera. The model and proportional feedback control make the described approach robust to several variations such as new targets, varying lighting conditions, diminution and uniform load. Furthermore, the model lends itself to be transferred to a new environment with minimal effort.