Continuous-Time Markov Chains and Applications
Continuous-Time Markov Chains and Applications

Author: G. George Yin

Publisher: Springer Science & Business Media

Published: 2012-11-14

Total Pages: 442

ISBN-13: 1461443466

DOWNLOAD EBOOK

This book gives a systematic treatment of singularly perturbed systems that naturally arise in control and optimization, queueing networks, manufacturing systems, and financial engineering. It presents results on asymptotic expansions of solutions of Komogorov forward and backward equations, properties of functional occupation measures, exponential upper bounds, and functional limit results for Markov chains with weak and strong interactions. To bridge the gap between theory and applications, a large portion of the book is devoted to applications in controlled dynamic systems, production planning, and numerical methods for controlled Markovian systems with large-scale and complex structures in the real-world problems. This second edition has been updated throughout and includes two new chapters on asymptotic expansions of solutions for backward equations and hybrid LQG problems. The chapters on analytic and probabilistic properties of two-time-scale Markov chains have been almost completely rewritten and the notation has been streamlined and simplified. This book is written for applied mathematicians, engineers, operations researchers, and applied scientists. Selected material from the book can also be used for a one semester advanced graduate-level course in applied probability and stochastic processes.

Continuous Time Markov Processes
Continuous Time Markov Processes

Author: Thomas Milton Liggett

Publisher: American Mathematical Soc.

Published: 2010

Total Pages: 290

ISBN-13: 0821849492

DOWNLOAD EBOOK

Markov processes are among the most important stochastic processes for both theory and applications. This book develops the general theory of these processes, and applies this theory to various special examples.

Introduction to Probability Models
Introduction to Probability Models

Author: Sheldon M. Ross

Publisher: Academic Press

Published: 2006-12-11

Total Pages: 801

ISBN-13: 0123756871

DOWNLOAD EBOOK

Introduction to Probability Models, Tenth Edition, provides an introduction to elementary probability theory and stochastic processes. There are two approaches to the study of probability theory. One is heuristic and nonrigorous, and attempts to develop in students an intuitive feel for the subject that enables him or her to think probabilistically. The other approach attempts a rigorous development of probability by using the tools of measure theory. The first approach is employed in this text. The book begins by introducing basic concepts of probability theory, such as the random variable, conditional probability, and conditional expectation. This is followed by discussions of stochastic processes, including Markov chains and Poison processes. The remaining chapters cover queuing, reliability theory, Brownian motion, and simulation. Many examples are worked out throughout the text, along with exercises to be solved by students. This book will be particularly useful to those interested in learning how probability theory can be applied to the study of phenomena in fields such as engineering, computer science, management science, the physical and social sciences, and operations research. Ideally, this text would be used in a one-year course in probability models, or a one-semester course in introductory probability theory or a course in elementary stochastic processes. New to this Edition: - 65% new chapter material including coverage of finite capacity queues, insurance risk models and Markov chains - Contains compulsory material for new Exam 3 of the Society of Actuaries containing several sections in the new exams - Updated data, and a list of commonly used notations and equations, a robust ancillary package, including a ISM, SSM, and test bank - Includes SPSS PASW Modeler and SAS JMP software packages which are widely used in the field Hallmark features: - Superior writing style - Excellent exercises and examples covering the wide breadth of coverage of probability topics - Real-world applications in engineering, science, business and economics

Markov Chains and Stochastic Stability
Markov Chains and Stochastic Stability

Author: Sean Meyn

Publisher: Cambridge University Press

Published: 2009-04-02

Total Pages: 623

ISBN-13: 0521731828

DOWNLOAD EBOOK

New up-to-date edition of this influential classic on Markov chains in general state spaces. Proofs are rigorous and concise, the range of applications is broad and knowledgeable, and key ideas are accessible to practitioners with limited mathematical background. New commentary by Sean Meyn, including updated references, reflects developments since 1996.

Continuous-Time Markov Decision Processes
Continuous-Time Markov Decision Processes

Author: Xianping Guo

Publisher: Springer Science & Business Media

Published: 2009-09-18

Total Pages: 240

ISBN-13: 3642025471

DOWNLOAD EBOOK

Continuous-time Markov decision processes (MDPs), also known as controlled Markov chains, are used for modeling decision-making problems that arise in operations research (for instance, inventory, manufacturing, and queueing systems), computer science, communications engineering, control of populations (such as fisheries and epidemics), and management science, among many other fields. This volume provides a unified, systematic, self-contained presentation of recent developments on the theory and applications of continuous-time MDPs. The MDPs in this volume include most of the cases that arise in applications, because they allow unbounded transition and reward/cost rates. Much of the material appears for the first time in book form.

Sensitivity Analysis: Matrix Methods in Demography and Ecology
Sensitivity Analysis: Matrix Methods in Demography and Ecology

Author: Hal Caswell

Publisher: Springer

Published: 2019-04-02

Total Pages: 308

ISBN-13: 3030105342

DOWNLOAD EBOOK

This open access book shows how to use sensitivity analysis in demography. It presents new methods for individuals, cohorts, and populations, with applications to humans, other animals, and plants. The analyses are based on matrix formulations of age-classified, stage-classified, and multistate population models. Methods are presented for linear and nonlinear, deterministic and stochastic, and time-invariant and time-varying cases. Readers will discover results on the sensitivity of statistics of longevity, life disparity, occupancy times, the net reproductive rate, and statistics of Markov chain models in demography. They will also see applications of sensitivity analysis to population growth rates, stable population structures, reproductive value, equilibria under immigration and nonlinearity, and population cycles. Individual stochasticity is a theme throughout, with a focus that goes beyond expected values to include variances in demographic outcomes. The calculations are easily and accurately implemented in matrix-oriented programming languages such as Matlab or R. Sensitivity analysis will help readers create models to predict the effect of future changes, to evaluate policy effects, and to identify possible evolutionary responses to the environment. Complete with many examples of the application, the book will be of interest to researchers and graduate students in human demography and population biology. The material will also appeal to those in mathematical biology and applied mathematics.

Cont Markov Chains
Cont Markov Chains

Author: Borkar

Publisher: CRC Press

Published: 1991-04-30

Total Pages: 196

ISBN-13: 9780582068216

DOWNLOAD EBOOK

Provides a novel treatment of many problems in controlled Markov chains based on occupation measures and convex analysis. Includes a rederivation of many classical results, a general treatment of the ergodic control problems and an extensive study of the asymptotic behavior of the self-tuning adaptive controller and its variant, the Kumar-Becker-Lin scheme. Also includes a novel treatment of some multiobjective control problems, inaccessible to traditional methods. Annotation copyrighted by Book News, Inc., Portland, OR

Brownian Motion, Martingales, and Stochastic Calculus
Brownian Motion, Martingales, and Stochastic Calculus

Author: Jean-François Le Gall

Publisher: Springer

Published: 2016-04-28

Total Pages: 282

ISBN-13: 3319310895

DOWNLOAD EBOOK

This book offers a rigorous and self-contained presentation of stochastic integration and stochastic calculus within the general framework of continuous semimartingales. The main tools of stochastic calculus, including Itô’s formula, the optional stopping theorem and Girsanov’s theorem, are treated in detail alongside many illustrative examples. The book also contains an introduction to Markov processes, with applications to solutions of stochastic differential equations and to connections between Brownian motion and partial differential equations. The theory of local times of semimartingales is discussed in the last chapter. Since its invention by Itô, stochastic calculus has proven to be one of the most important techniques of modern probability theory, and has been used in the most recent theoretical advances as well as in applications to other fields such as mathematical finance. Brownian Motion, Martingales, and Stochastic Calculus provides a strong theoretical background to the reader interested in such developments. Beginning graduate or advanced undergraduate students will benefit from this detailed approach to an essential area of probability theory. The emphasis is on concise and efficient presentation, without any concession to mathematical rigor. The material has been taught by the author for several years in graduate courses at two of the most prestigious French universities. The fact that proofs are given with full details makes the book particularly suitable for self-study. The numerous exercises help the reader to get acquainted with the tools of stochastic calculus.

Numerical Methods for Stochastic Control Problems in Continuous Time
Numerical Methods for Stochastic Control Problems in Continuous Time

Author: Harold Kushner

Publisher: Springer Science & Business Media

Published: 2013-11-27

Total Pages: 480

ISBN-13: 146130007X

DOWNLOAD EBOOK

Stochastic control is a very active area of research. This monograph, written by two leading authorities in the field, has been updated to reflect the latest developments. It covers effective numerical methods for stochastic control problems in continuous time on two levels, that of practice and that of mathematical development. It is broadly accessible for graduate students and researchers.

Discrete-Time Markov Chains
Discrete-Time Markov Chains

Author: George Yin

Publisher: Springer Science & Business Media

Published: 2005

Total Pages: 372

ISBN-13: 9780387219486

DOWNLOAD EBOOK

Focusing on discrete-time-scale Markov chains, the contents of this book are an outgrowth of some of the authors' recent research. The motivation stems from existing and emerging applications in optimization and control of complex hybrid Markovian systems in manufacturing, wireless communication, and financial engineering. Much effort in this book is devoted to designing system models arising from these applications, analyzing them via analytic and probabilistic techniques, and developing feasible computational algorithms so as to reduce the inherent complexity. This book presents results including asymptotic expansions of probability vectors, structural properties of occupation measures, exponential bounds, aggregation and decomposition and associated limit processes, and interface of discrete-time and continuous-time systems. One of the salient features is that it contains a diverse range of applications on filtering, estimation, control, optimization, and Markov decision processes, and financial engineering. This book will be an important reference for researchers in the areas of applied probability, control theory, operations research, as well as for practitioners who use optimization techniques. Part of the book can also be used in a graduate course of applied probability, stochastic processes, and applications.