Textbook in PDF format

With the fast development of big data and artificial intelligence, a natural question is how do we analyze data more efficiently ? One of the efficient ways is to use optimization. What is optimization ? Optimization exists everywhere. People optimize. As long as you have choices, you do optimization. Optimization is the key of operations research. This book introduces the basic definitions and theory about numerical optimization, including optimality conditions for unconstrained and constrained optimization, as well as algorithms for unconstrained and constrained problems. Moreover, it also includes the nonsmooth Newton’s method, which plays an important role in large-scale numerical optimization. Finally, based on the author’s research experiences, several latest applications about optimization are introduced, including optimization algorithms for hypergraph matching, support vector machine and bilevel optimization approach for hyperparameter selection in machine learning. With these optimization tools, one can deal with data more efficiently.
This book is based on the author ’ s lecture “Modern Optimization Methods” given to graduate students at the Beijing Institute of Technology since 2015. It aimed at presenting complete and systematic theories of numerical optimization and their latest applications in different areas, especially in machine learning, statistics, and computer science