Stochastic Programing

Covering deterministic scheduling, stochastic scheduling, and the probabilistic analysis of algorithms, this unusually broad view of the subject brings together tutorials, surveys and articles with or

"Engaging, elegantly written." — Applied Mathematical Modelling. A distinguished theoretical chemist and engineer discusses the types of models — finite, statistical, stochastic, and more — as well as

Explains how Hilbert space techniques cross the boundaries into the foundations of probability and statistics. Focuses on the theory of martingales stochastic integration, interpolation and density es

This book provides an easily accessible, computationally-oriented introduction into the numerical solution of stochastic differential equations using computer experiments. It develops in the reader an

Modern survival analysis and more general event history analysis may be effectively handled in the mathematical framework of counting processes, stochastic integration, martingale central limit theor

Concise advanced-level introduction to stochastic processes that frequently arise in applied probability. Largely self-contained text covers Poisson process, renewal theory, Markov chains, inventory

This book describes a powerful and flexible technique for the modeling of behavior, based on evolutionary principles. The technique employs stochastic dynamic programming and permits the analysis of b

Stochastic mechanics is a description of quantum phenomena in classicalprobabilistic terms. This work contains a detailed account of thekinematics of diffusion processes, including diffusions on curve

A high school student can create deep Q-learning code to control her robot, without any understanding of the meaning of 'deep' or 'Q', or why the code sometimes fails. This book is designed to explain the science behind reinforcement learning and optimal control in a way that is accessible to students with a background in calculus and matrix algebra. A unique focus is algorithm design to obtain the fastest possible speed of convergence for learning algorithms, along with insight into why reinforcement learning sometimes fails. Advanced stochastic process theory is avoided at the start by substituting random exploration with more intuitive deterministic probing for learning. Once these ideas are understood, it is not difficult to master techniques rooted in stochastic control. These topics are covered in the second part of the book, starting with Markov chain theory and ending with a fresh look at actor-critic methods for reinforcement learning.

In pioneering work in the 1950s, S. Karlin and J. McGregor showed that probabilistic aspects of certain Markov processes can be studied by analyzing orthogonal eigenfunctions of associated operators. In the decades since, many authors have extended and deepened this surprising connection between orthogonal polynomials and stochastic processes. This book gives a comprehensive analysis of the spectral representation of the most important one-dimensional Markov processes, namely discrete-time birth-death chains, birth-death processes and diffusion processes. It brings together the main results from the extensive literature on the topic with detailed examples and applications. Also featuring an introduction to the basic theory of orthogonal polynomials and a selection of exercises at the end of each chapter, it is suitable for graduate students with a solid background in stochastic processes as well as researchers in orthogonal polynomials and special functions who want to learn about appl

Mathematical models of bond markets are of interest to researchers working in applied mathematics, especially in mathematical finance. This book concerns bond market models in which random elements are represented by Lévy processes. These are more flexible than classical models and are well suited to describing prices quoted in a discontinuous fashion. The book's key aims are to characterize bond markets that are free of arbitrage and to analyze their completeness. Nonlinear stochastic partial differential equations (SPDEs) are an important tool in the analysis. The authors begin with a relatively elementary analysis in discrete time, suitable for readers who are not familiar with finance or continuous time stochastic analysis. The book should be of interest to mathematicians, in particular to probabilists, who wish to learn the theory of the bond market and to be exposed to attractive open mathematical problems.

At the intersection between statistical physics and rigorous econometric analysis, this powerful new framework sheds light on how innovation and competition shape the growth and decline of companies and industries. Analyzing various sources of data including a unique micro level database which collects historic data on the sales of more than 3,000 firms and 50,000 products in 20 countries, the authors introduce and test a model of innovation and proportional growth, which relies on minimal assumptions and accounts for the empirically observed regularities. Through a combination of extensive stochastic simulations and statistical tests, the authors investigate to what extent their simple assumptions are falsified by empirically observable facts. Physicists looking for application of their mathematical and modelling skills to relevant economic problems as well as economists interested in the explorative analysis of extensive data sets and in a physics-orientated way of thinking will find

Building on the Cambridge Element Agent Based Models of Social Life: Fundamentals (Cambridge, 2020), we move on to the next level. We do this by building agent based models of polarization and ethnocentrism. In the process, we develop: stochastic models, which add a crucial element of uncertainty to human interaction; models of human interactions structured by social networks; and 'evolutionary' models in which agents using more effective decision rules are more likely to survive and prosper than others. The aim is to leave readers with an effective toolkit for building, running and analyzing agent based modes of social interaction.

Many areas of continuum physics pose a challenge to physicists. What are the most general, admissible statistically homogeneous and isotropic tensor-valued random fields (TRFs)? Previously, only the TRFs of rank 0 were completely described. This book assembles a complete description of such fields in terms of one- and two-point correlation functions for tensors of ranks 1 through 4. Working from the standpoint of invariance of physical laws with respect to the choice of a coordinate system, spatial domain representations, as well as their wavenumber domain counterparts are rigorously given in full detail. The book also discusses, an introduction to a range of continuum theories requiring TRFs, an introduction to mathematical theories necessary for the description of homogeneous and isotropic TRFs, and a range of applications including a strategy for simulation of TRFs, ergodic TRFs, scaling laws of stochastic constitutive responses, and applications to stochastic partial differential e

Written in the intuitive yet rigorous style that readers of A Foundation in Digital Communication have come to expect, this second edition includes entirely new chapters on the radar problem (with Lyapunov's theorem) and intersymbol interference channels, new discussion of the baseband representation of passband noise, and a simpler, more geometric derivation of the optimal receiver for the additive white Gaussian noise channel. Other key topics covered include the definition of the power spectral density of nonstationary stochastic processes, the geometry of the space of energy-limited signals, the isometry properties of the Fourier transform, and complex sampling. Including over 500 homework problems and all the necessary mathematical background, this is the ideal text for one- or two-semester graduate courses on digital communications and courses on stochastic processes and detection theory. Solutions to problems and video lectures are available online.

From the basics to the forefront of modern research, this book presents all aspects of probability theory, statistics and data analysis from a Bayesian perspective for physicists and engineers. The book presents the roots, applications and numerical implementation of probability theory, and covers advanced topics such as maximum entropy distributions, stochastic processes, parameter estimation, model selection, hypothesis testing and experimental design. In addition, it explores state-of-the art numerical techniques required to solve demanding real-world problems. The book is ideal for students and researchers in physical sciences and engineering.

This rigorous, self-contained book describes mathematical and, in particular, stochastic and graph theoretic methods to assess the performance of complex networks and systems. It comprises three parts: the first is a review of probability theory; Part II covers the classical theory of stochastic processes (Poisson, Markov and queueing theory), which are considered to be the basic building blocks for performance evaluation studies; Part III focuses on the rapidly expanding new field of network science. This part deals with the recently obtained insight that many very different large complex networks – such as the Internet, World Wide Web, metabolic and human brain networks, utility infrastructures, social networks – evolve and behave according to general common scaling laws. This understanding is useful when assessing the end-to-end quality of Internet services and when designing robust and secure networks. Containing problems and solved solutions, the book is ideal for graduate student

Complexity science is the study of systems with many interdependent components. Such systems - and the self-organization and emergent phenomena they manifest - lie at the heart of many challenges of global importance. This book is a coherent introduction to the mathematical methods used to understand complexity, with plenty of examples and real-world applications. It starts with the crucial concepts of self-organization and emergence, then tackles complexity in dynamical systems using differential equations and chaos theory. Several classes of models of interacting particle systems are studied with techniques from stochastic analysis, followed by a treatment of the statistical mechanics of complex systems. Further topics include numerical analysis of PDEs, and applications of stochastic methods in economics and finance. The book concludes with introductions to space-time phases and selfish routing. The exposition is suitable for researchers, practitioners and students in complexity sci

The estimation of noisily observed states from a sequence of data has traditionally incorporated ideas from Hilbert spaces and calculus-based probability theory. As conditional expectation is the key concept, the correct setting for filtering theory is that of a probability space. Graduate engineers, mathematicians and those working in quantitative finance wishing to use filtering techniques will find in the first half of this book an accessible introduction to measure theory, stochastic calculus, and stochastic processes, with particular emphasis on martingales and Brownian motion. Exercises are included. The book then provides an excellent users' guide to filtering: basic theory is followed by a thorough treatment of Kalman filtering, including recent results which extend the Kalman filter to provide parameter estimates. These ideas are then applied to problems arising in finance, genetics and population modelling in three separate chapters, making this a comprehensive resource for b

This book is intended to provide the reader with a firm conceptual and empirical understanding of basic information-theoretic econometric models and methods. Because most data are observational, practitioners work with indirect noisy observations and ill-posed econometric models in the form of stochastic inverse problems. Consequently, traditional econometric methods in many cases are not applicable for answering many of the quantitative questions that analysts wish to ask. After initial chapters deal with parametric and semiparametric linear probability models, the focus turns to solving nonparametric stochastic inverse problems. In succeeding chapters, a family of power divergence measure-likelihood functions are introduced for a range of traditional and nontraditional econometric-model problems. Finally, within either an empirical maximum likelihood or loss context, Ron C. Mittelhammer and George G. Judge suggest a basis for choosing a member of the divergence family.

This book is intended to provide the reader with a firm conceptual and empirical understanding of basic information-theoretic econometric models and methods. Because most data are observational, practitioners work with indirect noisy observations and ill-posed econometric models in the form of stochastic inverse problems. Consequently, traditional econometric methods in many cases are not applicable for answering many of the quantitative questions that analysts wish to ask. After initial chapters deal with parametric and semiparametric linear probability models, the focus turns to solving nonparametric stochastic inverse problems. In succeeding chapters, a family of power divergence measure-likelihood functions are introduced for a range of traditional and nontraditional econometric-model problems. Finally, within either an empirical maximum likelihood or loss context, Ron C. Mittelhammer and George G. Judge suggest a basis for choosing a member of the divergence family.

From Measures to Itô Integrals gives a clear account of measure theory, leading via L2-theory to Brownian motion, Itô integrals and a brief look at martingale calculus. Modern probability theory and the applications of stochastic processes rely heavily on an understanding of basic measure theory. This text is ideal preparation for graduate-level courses in mathematical finance and perfect for any reader seeking a basic understanding of the mathematics underpinning the various applications of Itô calculus.

This is a textbook and reference for readers interested in quasilinear control (QLC). QLC is a set of methods for performance analysis and design of linear plant or nonlinear instrumentation (LPNI) systems. The approach of QLC is based on the method of stochastic linearization, which reduces the nonlinearities of actuators and sensors to quasilinear gains. Unlike the usual - Jacobian linearization - stochastic linearization is global. Using this approximation, QLC extends most of the linear control theory techniques to LPNI systems. A bisection algorithm for solving these equations is provided. In addition, QLC includes new problems, specific for the LPNI scenario. Examples include Instrumented LQR/LQG, in which the controller is designed simultaneously with the actuator and sensor, and partial and complete performance recovery, in which the degradation of linear performance is either contained by selecting the right instrumentation or completely eliminated by the controller boosting.

Nature is inherently noisy and nonlinear. It is noisy in the sense that all macroscopic systems are subject to the fluctuations of their environments and also to internal fluctuations. It is nonlinear in the sense that the restoring force on a system displaced from equilibrium does not usually vary linearly with the size of the displacement. To calculate the properties of stochastic (noisy) nonlinear systems is in general extremely difficult, although considerable progress has been made in the past. The three volumes that make up Noise in Nonlinear Dynamical Systems comprise a collection of specially written authoritative reviews on all aspects of the subject, representative of all the major practitioners in the field. The first volume deals with the basic theory of stochastic nonlinear systems. It includes an historical overview of the origins of the field, chapters covering some developed theoretical techniques for the study of coloured noise, and the first English-language translati

Nature is inherently noisy and nonlinear. It is noisy in the sense that all macroscopic systems are subject to the fluctuations of their environments and also to internal fluctuations. It is nonlinear in the sense that the restoring force on a system displaced from equilibrium does not usually vary linearly with the size of the displacement. To calculate the properties of stochastic (noisy) nonlinear systems is in general extremely difficult, although considerable progress has been made in the past. The three volumes that make up Noise in Nonlinear Dynamical Systems comprise a collection of specially written authoritative reviews on all aspects of the subject, representative of all the major practitioners in the field. The third volume deals with experimental aspects of the study of noise in nonlinear dynamical systems. It covers noise-driven phenomena in superfluid helium, liquid crystals, lasers and optical bistability as well as the solution of stochastic equations by digital simula

This rigourous and self-contained book describes mathematical and, in particular, stochastic methods to assess the performance of networked systems. It consists of three parts. The first part is a review on probability theory. Part two covers the classical theory of stochastic processes (Poisson, renewal, Markov and queuing theory), which are considered to be the basic building blocks for performance evaluation studies. Part three focuses on the relatively new field of the physics of networks. This part deals with the recently obtained insights that many very different large complex networks - such as the Internet, World Wide Web, proteins, utility infrastructures, social networks - evolve and behave according to more general common scaling laws. This understanding is useful when assessing the end-to-end quality of communications services, for example, in Internet telephony, real-time video and interacting games. Containing problems and solutions, this book is ideal for graduate studen

This book offers a complete, succinct account of the principles of financial derivatives pricing. The first chapter provides readers with an intuitive exposition of basic random calculus. Concepts such as volatility and time, random walks, geometric Brownian motion, and Ito's lemma are discussed heuristically. The second chapter develops generic pricing techniques for assets and derivatives, determining the notion of a stochastic discount factor or pricing kernel, and then uses this concept to price conventional and exotic derivatives. The third chapter applies the pricing concepts to the special case of interest rate markets, namely, bonds and swaps, and discusses factor models and term structure consistent models. The fourth chapter deals with a variety of mathematical topics that underlie derivatives pricing and portfolio allocation decisions such as mean-reverting processes and jump processes and discusses related tools of stochastic calculus such as Kolmogorov equations, martingal

The main theme of this book is the interplay between the behaviour of a class of stochastic processes (random walks) and discrete structure theory. The author considers Markov chains whose state space is equipped with the structure of an infinite, locally finite graph, or as a particular case, of a finitely generated group. The transition probabilities are assumed to be adapted to the underlying structure in some way that must be specified precisely in each case. From the probabilistic viewpoint, the question is what impact the particular type of structure has on various aspects of the behaviour of the random walk. Vice-versa, random walks may also be seen as useful tools for classifying, or at least describing the structure of graphs and groups. Links with spectral theory and discrete potential theory are also discussed. This book will be essential reading for all researchers working in stochastic process and related topics.

This book concerns the use of concepts from statistical physics in the description of financial systems. The authors illustrate the scaling concepts used in probability theory, critical phenomena, and fully developed turbulent fluids. These concepts are then applied to financial time series. The authors also present a stochastic model that displays several of the statistical properties observed in empirical data. Statistical physics concepts such as stochastic dynamics, short- and long-range correlations, self-similarity and scaling permit an understanding of the global behaviour of economic systems without first having to work out a detailed microscopic description of the system. Physicists will find the application of statistical physics concepts to economic systems interesting. Economists and workers in the financial world will find useful the presentation of empirical analysis methods and well-formulated theoretical tools that might help describe systems composed of a huge number o

A natural sequel to the author's previous book Combinatorial Matrix Theory written with H. J. Ryser, this is the first book devoted exclusively to existence questions, constructive algorithms, enumeration questions, and other properties concerning classes of matrices of combinatorial significance. Several classes of matrices are thoroughly developed including the classes of matrices of 0's and 1's with a specified number of 1's in each row and column (equivalently, bipartite graphs with a specified degree sequence), symmetric matrices in such classes (equivalently, graphs with a specified degree sequence), tournament matrices with a specified number of 1's in each row (equivalently, tournaments with a specified score sequence), nonnegative matrices with specified row and column sums, and doubly stochastic matrices. Most of this material is presented for the first time in book format and the chapter on doubly stochastic matrices provides the most complete development of the topic to dat

This is the first book presenting a stochastic extension of process algebra, PEPA; this is shown to be suitable for specifying a Markov process, which can then be applied to performance modelling. The method, which is illustrated with case studies taken from the area of communication systems, can readily be used to construct a variety of models that can be analysed using standard numerical techniques. One of the major advantages of PEPA over the standard methods for specifying stochastic performance models is the inherent apparatus for reasoning about the structure and behaviour of models. In the later chapters this apparatus is exploited to define four equivalence relations over PEPA components. Each of these notions of equivalence has intrinsic interest from a process algebra perspective. However, they are also demonstrated to be useful in a performance modelling context. To conclude the book, a section has been added surveying recent results in the area and discussing open questions

The estimation of noisily observed states from a sequence of data has traditionally incorporated ideas from Hilbert spaces and calculus-based probability theory. As conditional expectation is the key concept, the correct setting for filtering theory is that of a probability space. Graduate engineers, mathematicians and those working in quantitative finance wishing to use filtering techniques will find in the first half of this book an accessible introduction to measure theory, stochastic calculus, and stochastic processes, with particular emphasis on martingales and Brownian motion. Exercises are included. The book then provides an excellent users' guide to filtering: basic theory is followed by a thorough treatment of Kalman filtering, including recent results which extend the Kalman filter to provide parameter estimates. These ideas are then applied to problems arising in finance, genetics and population modelling in three separate chapters, making this a comprehensive resource for b

This book has two components: stochastic dynamics and stochastic random combinatorial analysis. The first discusses evolving patterns of interactions of a large but finite number of agents of several types. Changes of agent types or their choices or decisions over time are formulated as jump Markov processes with suitably specified transition rates: optimisations by agents make these rates generally endogenous. Probabilistic equilibrium selection rules are also discussed, together with the distributions of relative sizes of the bases of attraction. As the number of agents approaches infinity, we recover deterministic macroeconomic relations of more conventional economic models. The second component analyses how agents form clusters of various sizes. This has applications for discussing sizes or shares of markets by various agents which involve some combinatorial analysis patterned after the population genetics literature. These are shown to be relevant to distributions of returns to as

This book offers a complete, succinct account of the principles of financial derivatives pricing. The first chapter provides readers with an intuitive exposition of basic random calculus. Concepts such as volatility and time, random walks, geometric Brownian motion, and Ito's lemma are discussed heuristically. The second chapter develops generic pricing techniques for assets and derivatives, determining the notion of a stochastic discount factor or pricing kernel, and then uses this concept to price conventional and exotic derivatives. The third chapter applies the pricing concepts to the special case of interest rate markets, namely, bonds and swaps, and discusses factor models and term structure consistent models. The fourth chapter deals with a variety of mathematical topics that underlie derivatives pricing and portfolio allocation decisions such as mean-reverting processes and jump processes and discusses related tools of stochastic calculus such as Kolmogorov equations, martingal

Finance provides a dramatic example of the successful application of advanced mathematical techniques to the practical problem of pricing financial derivatives. This self-contained 2002 text is designed for first courses in financial calculus aimed at students with a good background in mathematics. Key concepts such as martingales and change of measure are introduced in the discrete time framework, allowing an accessible account of Brownian motion and stochastic calculus: proofs in the continuous-time world follow naturally. The Black-Scholes pricing formula is first derived in the simplest financial context. The second half of the book is then devoted to increasing the financial sophistication of the models and instruments. The final chapter introduces more advanced topics including stock price models with jumps, and stochastic volatility. A valuable feature is the large number of exercises and examples, designed to test technique and illustrate how the methods and concepts can be app

Finance provides a dramatic example of the successful application of advanced mathematical techniques to the practical problem of pricing financial derivatives. This self-contained 2002 text is designed for first courses in financial calculus aimed at students with a good background in mathematics. Key concepts such as martingales and change of measure are introduced in the discrete time framework, allowing an accessible account of Brownian motion and stochastic calculus: proofs in the continuous-time world follow naturally. The Black-Scholes pricing formula is first derived in the simplest financial context. The second half of the book is then devoted to increasing the financial sophistication of the models and instruments. The final chapter introduces more advanced topics including stock price models with jumps, and stochastic volatility. A valuable feature is the large number of exercises and examples, designed to test technique and illustrate how the methods and concepts can be app

This book has two components: stochastic dynamics and stochastic random combinatorial analysis. The first discusses evolving patterns of interactions of a large but finite number of agents of several types. Changes of agent types or their choices or decisions over time are formulated as jump Markov processes with suitably specified transition rates: optimisations by agents make these rates generally endogenous. Probabilistic equilibrium selection rules are also discussed, together with the distributions of relative sizes of the bases of attraction. As the number of agents approaches infinity, we recover deterministic macroeconomic relations of more conventional economic models. The second component analyses how agents form clusters of various sizes. This has applications for discussing sizes or shares of markets by various agents which involve some combinatorial analysis patterned after the population genetics literature. These are shown to be relevant to distributions of returns to as

The main theme of this book is the interplay between the behaviour of a class of stochastic processes (random walks) and discrete structure theory. The author considers Markov chains whose state space is equipped with the structure of an infinite, locally finite graph, or as a particular case, of a finitely generated group. The transition probabilities are assumed to be adapted to the underlying structure in some way that must be specified precisely in each case. From the probabilistic viewpoint, the question is what impact the particular type of structure has on various aspects of the behaviour of the random walk. Vice-versa, random walks may also be seen as useful tools for classifying, or at least describing the structure of graphs and groups. Links with spectral theory and discrete potential theory are also discussed. This book will be essential reading for all researchers working in stochastic process and related topics.

This book concerns the use of concepts from statistical physics in the description of financial systems. The authors illustrate the scaling concepts used in probability theory, critical phenomena, and fully developed turbulent fluids. These concepts are then applied to financial time series. The authors also present a stochastic model that displays several of the statistical properties observed in empirical data. Statistical physics concepts such as stochastic dynamics, short- and long-range correlations, self-similarity and scaling permit an understanding of the global behaviour of economic systems without first having to work out a detailed microscopic description of the system. Physicists will find the application of statistical physics concepts to economic systems interesting. Economists and workers in the financial world will find useful the presentation of empirical analysis methods and well-formulated theoretical tools that might help describe systems composed of a huge number o