Συμβολή στην ανάπτυξη πολυπρακτορικής αρχιτεκτονικής αναπτυξιακού ρομποτικού ελέγχου στη βάση ασαφούς ενισχυτικής μάθησης: εφαρμογή στον επιδέξιο ρομποτικό χειρισμό

John Karigiannis

Abstract: This thesis proposes a model-free learning mechanism based on a nested hierarchical multi-agent architecture, which is applied in the context of dexterous robot manipulation control. In the proposed multi-agent system, each agent forms a local (partial) view of the global system state and task progress, through a recursive (top-down/bottom-up) learning process. By organizing the agents in a nested architecture, the goal is to facilitate modular scaling to more complex kinematic topologies, with loose control coupling among the agents. Reinforcement learning is applied within each agent, to evolve a local state-to-action mapping in a continuous domain, thus leading to a system that exhibits developmental properties. The agents correspond in fact to independent degrees-of-freedom (DOF) of the system, managing to gain experience over the task that they collaboratively perform by continuously exploring and exploiting their state-to-action mapping space. This thesis addresses problem settings in the domain of kinematic control of dexterous robot manipulation. Three sets of numerical experiments are performed: (i) the first one considers the case of a single-linkage open kinematic chain, presenting kinematic redundancies given the desired task-goal, (ii) the second experiment extends further on the previous case by considering three individual kinematic chains cooperatively acting to achieve a quasi-static multifinger grasp, and (iii) the last experiment extends the proposed multi-agent framework to a control problem in the field of autonomous mobile robots, by considering two e-Puck robots performing a collaborative “box-pushing” task. The focal issue in all experiments is to assess the capacity of the proposed multi-agent system to progressively and autonomously acquire cooperative sensorimotor skills through a self-learning process, that is, without the use of any explicit model-based planning strategy. Generalization and robustness properties of the overall multi-agent system are also explored. Furthermore, these experiments aim to demonstrate the scaling properties of the proposed nested-hierarchical architecture, where new higher-level agents can be recursively added in the hierarchy to encapsulate individual active DOFs. The experimental results presented in this thesis demonstrate the feasibility of such a distributed multi-agent control framework, showing that the solutions which emerge are plausible and near-optimal.