Food Arrangement Framework for Cooking Robots

Hoshito Nagahama, Ixchel Georgina Ramirez-Alpizar, Kensuke Harada

Abstract: We propose a food arrangement framework for a robot to automatically serve meals. We start from the premise that anybody knows how to arrange food. Based on this, we use a Convolutional Neural Network (CNN) to evaluate how good a food arrangement is. The CNN is trained using a dataset gathered through Amazon Mechanical Turk (AMT), where people are asked to choose the best food arrangement between a pair of pictures. The food arrangement and rearrangement is done entirely virtual through image processing. The initial food placement is random and evaluated by the CNN. If this evaluation is under a given threshold, the position of some ingredients will be changed according to the rearrangement algorithm we developed. For this algorithm we tested two different strategies for finding a relocation together with two different approaches for deciding which food to relocate. After the rearrangement is done, the CNN will evaluate again the food arrangement. The previous steps will be repeated until the food arrangement evaluation is beyond the given threshold. The resulting arrangement will be given to the robot for its actual execution. We evaluated our framework using two different sets of meals. We demonstrate that a UR3 robot is capable of serving a steak meal using a spatula-like end-effector.