Implicit Optimization: Models and Methods

We present a new class of optimization models, implicit optimization, which includes deep learning, nonlinear control, and mixed-integer programming as special cases. Implicit optimization provides a unified perspective on these different fields, leading to new algorithms and surprising connections. We propose two tractable algorithms to solve such problems based on their implicit equation structure: implicit gradient descent and the Fenchel alternative direction method of multipliers. We illustrate our theory and methods with numerical experiments.