Neural-Fields in Numerical Optimization with L4CasADi

Tim Salzmann, Jon Arrizabalaga, Lukas Pries, Markus Ryll

Keywords: optimization; machine learning; control systems; data-driven control

TL;DR: With L4CasADi Neural-Fields can easily be used in numerical optimization for robotic applications.

Abstract:

While real-world problems are often challenging to analyze analytically, deep learning excels in modeling complex processes from data. Existing optimization frameworks like CasADi facilitate seamless usage of solvers but face challenges when integrating learned process models into numerical optimizations. To address this gap, we present the Learning for CasADi (L4CasADi) framework, enabling the seamless integration of PyTorch-learned models with CasADi for efficient and potentially hardware-accelerated numerical optimization. The applicability of L4CasADi is demonstrated with two robotic tutorial examples: First, we optimize a fish's trajectory in a turbulent river for energy efficiency where the turbulent flow is represented by a PyTorch model. Second, we demonstrate how an implicit Neural Radiance Field environment representation can be easily leveraged for optimal control with L4CasADi.
L4CasADi is available under MIT license at https://github.com/Tim-Salzmann/l4casadi

Submission Number: 4