Stochastic Recursive Algorithms for Optimization—Simultaneous Perturbation Methods

Stochastic Recursive Algorithms for Optimization presents algorithms for constrained and unconstrained optimization and for reinforcement learning. Efficient perturbation approaches form a thread unifying all the algorithms considered. Simultaneous perturbation stochastic approximation and smooth fractional estimators for gradient- and Hessian-based methods are presented. These algorithms: ‧ are easily implemented; ‧ do not require an explicit system model; and ‧ work with real or simulated data. Chapters on their application in service systems, vehicular traffic control and communications networks illustrate this point. The book is self-contained with necessary mathematical results placed in an appendix. The text provides easy-to-use, off-the-shelf algorithms that are given detailed mathematical treatment so the material presented will be of significant interest to practitioners, academic researchers and graduate students alike. The breadth of applications makes the book appropriate for reader from similarly diverse backgrounds: workers in relevant areas of computer science, control engineering, management science, applied mathematics, industrial engineering and operations research will find the content of value.

All three authors have been extensively working in the area of stochastic control and optimization. S. Bhatnagar has worked for nearly 20 years in this area and has published extensively in both journals and conferences. This book in many ways summarizes the various strands of research that S.Bhatnagar has been involved in over the last decade. H.L.Prasad and Prashanth L.A. have been working in this area for over five years now and have been actively involved in various aspects of the research reported here. The entire book, in many ways, is a collection of the various strands of the research that has been primarily carried out by the authors themselves during the course of the last several years.