# Agile Flight with Optimization Embedded Networks

Under Review

## Abstract

To bridge the gap between perception and planning in traditional navigation systems, we address the challenge of learning optimal trajectories directly from depth information in an end-to-end fashion. Using neural networks as black-box replacements for traditional modules can compromise robustness and stability. Moreover, such methods often fail to adequately account for the robot's kinematic constraints, leading to trajectories that may not be satisfactorily executable. In this paper, we integrate the strengths of conventional methods and neural networks by introducing an optimization-embedded neural network based on a compact trajectory library. Neural networks establish spatial constraints for model-based trajectory planning, followed by robust numerical optimization to achieve feasible and optimal solutions. By making the process differentiable, our model seamlessly approximates the optimal trajectory. Additionally, the introduction of a regularized trajectory library enables the method to efficiently capture the spatial distribution of optimal trajectories with minimal storage cost, ensuring multimodal planning characteristics. Evaluations in complex, unseen environments demonstrate our method’s superior performance over state-of-the-art algorithms. Real-world flight experiments with a small onboard computer showcase the quadrotor’s ability to navigate swiftly through dense forests.

## Simulation Experiments

## (playing speed: 1x)

## Benchmark Comparisons

## Maximal speed: 8 m/s; Maximal acceleration: 10 m/s2

Ego-method

I-planner

Our method

## Real-World Experiments

## (playing speed: 1x)

## Wooden Area

## Maximal speed: 5 m/s; Maximal acceleration: 6 m/s2

Case 0

Case 1

Case 2

Case 3

Case 4

Case 5

Case 6

Case 7

Case 8

## Flying with Blurred Camera (maximal speed: 5 m/s, maximal acceleration: 6 m/s2)

## Maximal speed: 5 m/s; Maximal acceleration: 6 m/s2