Stochastic Recursive Algorithms for Optimization

Optimization techniques are at the core of data science, including data analysis and machine learning. An understanding of basic optimization techniques and their fundamental properties provides important grounding for students, researchers, and practitioners in these areas. This text covers the fundamentals of optimization algorithms in a compact, self-contained way, focusing on the techniques most relevant to data science. An introductory chapter demonstrates that many standard problems in data science can be formulated as optimization problems. Next, many fundamental methods in optimization are described and analyzed, including: gradient and accelerated gradient methods for unconstrained optimization of smooth (especially convex) functions; the stochastic gradient method, a workhorse algorithm in machine learning; the coordinate descent approach; several key algorithms for constrained optimization problems; algorithms for minimizing nonsmooth functions arising in data science; found

Based on the author's forty years of teaching experience, this unique textbook covers both basic and advanced concepts of optimization theory and methods for process systems engineers. Topics covered include continuous, discrete and logic optimization (linear, nonlinear, mixed-integer and generalized disjunctive programming), optimization under uncertainty (stochastic programming and flexibility analysis), and decomposition techniques (Lagrangean and Benders decomposition). Assuming only a basic background in calculus and linear algebra, it enables easy understanding of mathematical reasoning, and numerous examples throughout illustrate key concepts and algorithms. End-of-chapter exercises involving theoretical derivations and small numerical problems, as well as in modeling systems like GAMS, enhance understanding and help put knowledge into practice. Accompanied by two appendices containing web links to modeling systems and models related to applications in PSE, this is an essential