Abstract

We propose new methods for learning control policies and neural network Lyapunov functions for nonlinear control problems, with provable guarantee of stability. The framework consists of a learner that attempts to find the control and Lyapunov functions, and a falsifier that finds counterexamples to quickly guide the learner towards solutions. The procedure terminates when no counterexample is found by the falsifier, in which case the controlled nonlinear system is provably stable. The approach significantly simplifies the process of Lyapunov control design, provides end-to-end correctness guarantee, and can obtain much larger regions of attraction than existing methods such as LQR and SOS/SDP. We show experiments on how the new methods obtain high-quality solutions for challenging control problems.

Overall algorithm:

3-Link planar robot balancing with learned controller:

Published in NeurIPS 2019 [Paper] [Appendices] [Poster] [Code]

Example Results:

Inverted pendulum:

Path tracking:

Caltech ducted fan in hover mode:

3-Link Planar Robot Balancing:

Bibtex

@inproceedings{NEURIPS2019_2647c1db,
 author = {Chang, Ya-Chien and Roohi, Nima and Gao, Sicun},
 booktitle = {Advances in Neural Information Processing Systems},
 editor = {H. Wallach and H. Larochelle and A. Beygelzimer and F. d\textquotesingle Alch\'{e}-Buc and E. Fox and R. Garnett},
 pages = {},
 publisher = {Curran Associates, Inc.},
 title = {Neural Lyapunov Control},
 url = {https://proceedings.neurips.cc/paper/2019/file/2647c1dba23bc0e0f9cdf75339e120d2-Paper.pdf},
 volume = {32},
 year = {2019}
}