Markov Processes

Piecewise-deterministic Markov processes form a class of stochastic models with a sizeable scope of applications: biology, insurance, neuroscience, networks, finance... Such processes are defined by a

The purpose of this monograph is to provide a theory of Markov processes that are invariant under the actions of Lie groups, focusing on ways to represent such processes in the spirit of the classical

The series is devoted to the publication of monographs and high-level textbooks in mathematics, mathematical methods and their applications. Apart from covering important areas of current interest, a

The book presents new methods of asymptotic analysis for nonlinearly perturbed semi-Markov processes with a finite phase space. These methods are based on special time-space screening procedures for s

Covering formulation, algorithms, and structural results, and linking theory to real-world applications in controlled sensing (including social learning, adaptive radars and sequential detection), this book focuses on the conceptual foundations of partially observed Markov decision processes (POMDPs). It emphasizes structural results in stochastic dynamic programming, enabling graduate students and researchers in engineering, operations research, and economics to understand the underlying unifying themes without getting weighed down by mathematical technicalities. Bringing together research from across the literature, the book provides an introduction to nonlinear filtering followed by a systematic development of stochastic dynamic programming, lattice programming and reinforcement learning for POMDPs. Questions addressed in the book include: when does a POMDP have a threshold optimal policy? When are myopic policies optimal? How do local and global decision makers interact in adaptive

An extension problem (often called a boundary problem) of Markov processes has been studied, particularly in the case of one-dimensional diffusion processes, by W. Feller, K. Itô, and H. P. McKean, am

Mark H.A. Davis introduced the Piecewise-Deterministic Markov Process (PDMP) class of stochastic hybrid models in an article in 1984. Today it is used to model a variety of complex systems in the fiel

This text fills a niche between a calculus-based probability course and a stochastic processes course typically taken by upper undergraduate students. While Markov processes are touched on in probabil

A careful and accessible exposition of functional analytic methods in stochastic analysis is provided in this book. It focuses on the interrelationship between three subjects in analysis: Markov proce

Semi-Markov Processes: Applications in System Reliability and Maintenance is a modern view of discrete state space and continuous time semi-Markov processes and their applications in reliability and m

This book explores important aspects of Markov and hidden Markov processes and the applications of these ideas to various problems in computational biology. The book starts from first principles, so t

This book provides a rigorous but elementary introduction to the theory of Markov Processes on a countable state space. It should be accessible to students with a solid undergraduate background in mat

This volume of the Selected Papers from Portugal is a product of the Seventeenth Congress of the Portuguese Statistical Society, held at the beautiful resort seaside city of Sesimbra, Portugal, from S

Revising his unpublished lecture notes, Oshima (U. of Erlangen-Nurnberg) extends results for symmetric Dirichlet forms to lower bounded semi-Dirichlet forms and to time-dependent semi-Dirichlet forms.