Effective assembly path planning plays a crucial role in various applications. However, the planning often encounter challenges due to narrow passages, where parts can easily get stuck or face difficulty in navigating through. From an insight that feasible initialization can play a significant role in path planning, we propose an initialization scheme using variable lifting. In this approach, the positions of the static part are treated as adjustable variables, allowing us to effectively widen narrow passages and facilitating the feasible initialization. Furthermore, physics simulation-based path refinement is proposed to efficiently resolve the problem while maintaining the minimal penetration during the solution process. Several examples are implemented to demonstrate how the proposed framework addresses challenging assembly path planning problems.

Assembly Path Planning via Variable Lifting and Physics Simulation

We propose a two-stage framework that integrates a learning-based estimator and a controller to effectively manage contact-intensive tasks. The estimator utilizes a Bayesian particle filter combined with a Mixture Density Network (MDN) structure, which is proficient at solving non-injective issues from contact data. The controller merges self-supervised and reinforcement learning (RL) methods, separating the parameters of the low-level admittance controller into labeled and unlabeled parameters. To improve reliability and generalization capabilities, a transformer model is utilized in self-supervised learning. The suggested framework is validated on a bolting task using a precise real-time simulator and is successfully applied in an experimental setting.

A Learning-Based Assembly Framework for Contact-Intensive Tight-Tolerance Tasks

Assembly processes involving humans and robots are challenging scenarios because the individual activities and access to shared workspace have to be coordinated. Fixed robot programs leave no room to diverge from a fixed protocol. Working on such a process can be very stressful for the user and lead to ineffective behavior or failure. To this end, we propose a novel approach of utilizing uncertainty-aware constraint-based scheduling in a reactive execution control framework facilitating behavior trees. This allows the robot to adapt to uncertain events such as delayed activity completions and activity selection (by the human). The user can rest assured that the robot will wait for him to finish his activity and will adapt to best complement the human-selected activities to complete the common task. In addition to the improved working conditions, our algorithm also leads to increased efficiency, even in scenarios with increased uncertainty. We evaluate our algorithm using a probabilistic simulation study and initial real robot experiments using a Franka Emika Panda robot and human tracking based on HTC Vive VR gloves. 

Online Scheduling and Reactive Behaviors for Effective Human-Robot-Collaboration

Peg-in-hole assembly of tightly fitting parts often requires multiple attempts. Parts need to be put together by performing a wiggling motion of undetermined length and can get stuck, requiring a restart. Recognizing unsuccessful insertion attempts early can help in reducing the \emph{makespan} of the assembly. This can be achieved by analyzing time-series data from force and torque measurements. We describe a transformer neural network model that is three times faster, i.e. requiring much shorter time series, for predicting failure than a dilated fully convolutional neural network. Albeit the transformer provides predictions with higher confidence, it does so at reduced accuracy. Yet, being able to call unsuccessful attempts early, makespan can be reduced by almost 40% which we show using a dataset with force-torque data from 241 peg-in-hole assembly runs with known outcomes. 

Early failure prediction during robotic assembly using Transformers

Machine Learning (ML) models in Robotic Assembly Sequence Planning (RASP) need to be introspective on the predicted solutions, i.e. whether they are feasible or not, to circumvent potential efficiency degradation. While previous works tackling this problem are in need of both feasible and infeasible examples, which is hard to prepare sufficiently in practice, in this work we propose a density-based feasibility learning method that requires only feasible examples. Concretely, we formulate the feasibility learning problem as Out-of-Distribution (OOD) detection with Normalizing Flows (NF), which are powerful generative models for estimating complex probability distributions. Empirically, the proposed method is demonstrated on robotic assembly use cases and outperforms other baselines in detecting infeasible assemblies. We further investigate the internal working mechanism of our method and show that a large memory savings can be obtained based on an advanced variant of NF.

RSS 2023 Workshop Robotic Assembly Submissions

Assembly Path Planning via Variable Lifting and Physics Simulation

A Learning-Based Assembly Framework for Contact-Intensive Tight-Tolerance Tasks

Online Scheduling and Reactive Behaviors for Effective Human-Robot-Collaboration

Early failure prediction during robotic assembly using Transformers

Density-based Feasibility Learning with Normalizing Flows for Introspective Robotic Assembly