DiLQR: Differentiable Iterative Linear Quadratic Regulator
Keywords: Differentiable control;iLQR;IL
Abstract: Differentiable control promises end-to-end differentiability and adaptability, effectively combining the advantages of both model-free and model-based control
approaches. However, the iterative Linear Quadratic Regulator (iLQR), despite
being a powerful nonlinear controller, still lacks differentiable capabilities. The
scalability of differentiating through extended iterations and horizons poses signifi
cant challenges, hindering iLQR from being an effective differentiable controller.
This paper introduces a framework that facilitates differentiation through iLQR,
allowing it to serve as a trainable and differentiable module, either as or within
a neural network. for control purposes. A novel aspect of this framework is the
analytical solution that it provides for the gradient of an iLQR controller through
implicit differentiation, which ensures a constant backward cost regardless of iteration, while producing an accurate gradient. We evaluate our framework on
imitation tasks on famous control benchmarks. Our analytical method demonstrates superior computational performance, achieving up to $\mathbf{128x}$ speedup and
a minimum of $\mathbf{21x}$ speedup compared to automatic differentiation. Our method
also demonstrates superior learning performance ($\mathbf{10^6}$x) compared to traditional
neural network policies and better model loss with differentiable controllers that
lack exact analytical gradients. Furthermore, we integrate our module into a
larger network with visual inputs to demonstrate the capacity of our method for
high-dimensional, fully end-to-end tasks. Codes can be found on the project
homepage https://sites.google.com/view/dilqr/.
Primary Area: applications to robotics, autonomy, planning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 12970
Loading